public inbox for [email protected]
help / color / mirror / Atom feedRe: Vacuum statistics
77+ messages / 16 participants
[nested] [flat]
* Re: Vacuum statistics
@ 2024-08-15 08:49 Alena Rybakina <[email protected]>
3 siblings, 2 replies; 77+ messages in thread
From: Alena Rybakina @ 2024-08-15 08:49 UTC (permalink / raw)
To: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; +Cc: pgsql-hackers; [email protected]
Hi!
On 13.08.2024 16:18, Ilia Evdokimov wrote:
>
> On 10.8.24 22:37, Andrei Zubkov wrote:
>
>> Hi, Ilia!
>>
>>> Do you consider not to create new table in pg_catalog but to save
>>> statistics in existing table? I mean pg_class or
>>> pg_stat_progress_analyze, pg_stat_progress_vacuum?
>>>
>> Thank you for your interest on our patch!
>>
>> *_progress views is not our case. They hold online statistics while
>> vacuum is in progress. Once work is done on a table the entry is gone
>> from those views. Idea of this patch is the opposite - it doesn't
>> provide online statistics but it accumulates statistics about rosources
>> consumed by all vacuum passes over all relations. It's much closer to
>> the pg_stat_all_tables than pg_stat_progress_vacuum.
>>
>> It seems pg_class is not the right place because it is not a statistic
>> view - it holds the current relation state and haven't anything about
>> the relation workload.
>>
>> Maybe the pg_stat_all_tables is the right place but I have several
>> thoughts about why it is not:
>> - Some statistics provided by this patch is really vacuum specific. I
>> don't think we want them in the relation statistics view.
>> - Postgres is extreamly extensible. I'm sure someday there will be
>> table AMs that does not need the vacuum at all.
>>
>> Right now vacuum specific workload views seems optimal choice to me.
>>
>> Regards,
>
>
> Agreed. They are not god places to store such statistics.
>
>
> I have some suggestions:
>
> 1. pgstatfuncs.c in functions tuplestore_put_for_database() and
> tuplestore_put_for_relation you can remove 'nulls' array if you're
> sure that columns cannot be NULL.
>
We need to use this for tuplestore_putvalues function. With this
function, we fill the table with the values of the statistics.
>
> 1.
>
>
> 2. These functions are almost the same and I would think of writing
> one function depending of type 'ExtVacReportType'
>
I'm not sure that I fully understand what you mean. Can you explain it
more clearly, please?
On 13.08.2024 16:37, Ilia Evdokimov wrote:
> And I have one suggestion for pg_stat_vacuum_database: I suppose we
> should add database's name column after 'dboid' column because it is
> difficult to read statistics without database's name. We could call it
> 'datname' just like in 'pg_stat_database' view.
>
Thank you. Fixed.
--
Regards,
Alena Rybakina
Postgres Professional:http://www.postgrespro.com
The Russian Postgres Company
Attachments:
[text/x-patch] v5-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (66.1K, 3-v5-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From ce8377fb8166da3636a73201d60dec06f46655b1 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 15 Aug 2024 11:30:13 +0300
Subject: [PATCH 1/4] Machinery for grabbing an extended vacuum statistics on
heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 159 +++++++++++++-
src/backend/access/heap/visibilitymap.c | 13 ++
src/backend/catalog/system_views.sql | 54 +++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 92 +++++---
src/backend/utils/activity/pgstat_relation.c | 35 ++-
src/backend/utils/adt/pgstatfuncs.c | 186 ++++++++++++++++
src/backend/utils/error/elog.c | 13 ++
src/include/catalog/pg_proc.dat | 10 +-
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 84 +++++++-
src/include/utils/elog.h | 2 +-
src/include/utils/pgstat_internal.h | 36 +++-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 51 +++++
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/expected/rules.out | 34 +++
.../expected/vacuum_tables_statistics.out | 200 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 158 ++++++++++++++
22 files changed, 1155 insertions(+), 44 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d82aa3d4896..3941ae26f2d 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -167,6 +167,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -194,6 +195,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -226,6 +229,22 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Cut-off values of parameters which changes implicitly during a vacuum
+ * process.
+ * Vacuum can't control their values, so we should store them before and after
+ * the processing.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz time;
+ PGRUsage ru;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -279,6 +298,115 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+ PGRUsage ru0;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ pg_rusage_init(&ru0);
+ starttime = GetCurrentTimestamp();
+
+ counters->ru = ru0;
+ counters->time = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+ PGRUsage ru1;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->time, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ /*
+ * Get difference of a system time and user time values in milliseconds.
+ * Use floating point representation to show tails of time diffs.
+ */
+ pg_rusage_init(&ru1);
+ report->system_time =
+ (ru1.ru.ru_stime.tv_sec - counters->ru.ru.ru_stime.tv_sec) * 1000. +
+ (ru1.ru.ru_stime.tv_usec - counters->ru.ru.ru_stime.tv_usec) * 0.001;
+ report->user_time =
+ (ru1.ru.ru_utime.tv_sec - counters->ru.ru.ru_utime.tv_sec) * 1000. +
+ (ru1.ru.ru_utime.tv_usec - counters->ru.ru.ru_utime.tv_usec) * 0.001;
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -311,6 +439,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -329,7 +459,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -346,6 +476,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -413,6 +544,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -574,6 +707,19 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -588,7 +734,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1380,6 +1527,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2277,11 +2426,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -3122,6 +3273,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3137,6 +3290,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33c..d72cade60a4 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,18 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Initially, it didn't matter what type of flags (all-visible or frozen) we received,
+ * we just performed a reverse concatenation operation. But this information is very important
+ * for vacuum statistics. We need to find out this usingthe bit concatenation operation
+ * with the VISIBILITYMAP_ALL_VISIBLE and VISIBILITYMAP_ALL_FROZEN masks,
+ * and where the desired one matches, we increment the value there.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 19cabc9a47f..68e6bfe6115 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1371,3 +1371,57 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stat_vacuum_tables(db.oid, rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace;
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 7d8e9d20454..363924d00db 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -103,6 +103,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2394,6 +2397,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 22c057fe61b..13ab633086a 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1043,6 +1043,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index b2ca3f39b7a..6a788f2b586 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -146,34 +146,6 @@
#define PGSTAT_FILE_ENTRY_HASH 'S' /* stats entry identified by
* PgStat_HashKey */
-/* hash table for statistics snapshots entry */
-typedef struct PgStat_SnapshotEntry
-{
- PgStat_HashKey key;
- char status; /* for simplehash use */
- void *data; /* the stats data itself */
-} PgStat_SnapshotEntry;
-
-
-/* ----------
- * Backend-local Hash Table Definitions
- * ----------
- */
-
-/* for stats snapshot entries */
-#define SH_PREFIX pgstat_snapshot
-#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
-#define SH_KEY_TYPE PgStat_HashKey
-#define SH_KEY key
-#define SH_HASH_KEY(tb, key) \
- pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
-#define SH_EQUAL(tb, a, b) \
- pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
-#define SH_SCOPE static inline
-#define SH_DEFINE
-#define SH_DECLARE
-#include "lib/simplehash.h"
-
/* ----------
* Local function forward declarations
@@ -190,7 +162,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -830,6 +802,40 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
+void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->system_time += src->system_time;
+ dst->user_time += src->user_time;
+ dst->total_time += src->total_time;
+ dst->interrupts += src->interrupts;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->dead_tuples += src->dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+}
/* ------------------------------------------------------------
* Fetching of stats
@@ -896,7 +902,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, Oid objoid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1012,7 +1018,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1062,8 +1068,30 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 8a3f7d434cf..d40d43cdb4a 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -204,12 +204,40 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.interrupts++;
+ pgstat_unlock_entry(entry_ref);
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -233,6 +261,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -861,6 +891,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 32211371237..f94e562009d 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -31,6 +31,7 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+#include "utils/pgstat_internal.h"
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -2032,3 +2033,188 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objoid));
}
+
+#define EXTVACHEAPSTAT_COLUMNS 27
+
+static Oid CurrentDatabaseId = InvalidOid;
+
+
+/*
+ * Fetch stat collector data for specific database and table, which loading from disc.
+ * It is maybe expensive, but i guess we won't use that machinery often.
+ * The kind of bufferization is based on CurrentDatabaseId value.
+ */
+static PgStat_StatTabEntry *
+fetch_dbstat_tabentry(Oid dbid, Oid relid)
+{
+ Oid storedMyDatabaseId = MyDatabaseId;
+ PgStat_StatTabEntry *tabentry = NULL;
+
+ if (OidIsValid(CurrentDatabaseId) && CurrentDatabaseId == dbid)
+ /* Quick path when we read data from the same database */
+ return pgstat_fetch_stat_tabentry(relid);
+
+ pgstat_clear_snapshot();
+
+ /* Tricky turn here: enforce pgstat to think that our database has dbid */
+
+ MyDatabaseId = dbid;
+
+ PG_TRY();
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ MyDatabaseId = storedMyDatabaseId;
+ }
+ PG_CATCH();
+ {
+ MyDatabaseId = storedMyDatabaseId;
+ }
+ PG_END_TRY();
+
+ return tabentry;
+}
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_StatTabEntry *tabentry, int ncolumns)
+{
+ Datum values[EXTVACHEAPSTAT_COLUMNS];
+ bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_fetched -
+ tabentry->vacuum_ext.blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, tabentry->vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(tabentry->vacuum_ext.interrupts);
+
+ Assert(i == ncolumns);
+
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+/*
+ * Get the vacuum statistics for the heap tables or indexes.
+ */
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_StatTabEntry *tabentry;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* Check if caller supports us returning a tuplestore */
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("set-valued function called in context that cannot accept a set")));
+ /* Switch to long-lived context to create the returned data structures */
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ Assert(tupdesc->natts == ncolumns);
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ Assert (tupstore != NULL);
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Load table statistics for specified database. */
+ if (OidIsValid(relid))
+ {
+ tabentry = fetch_dbstat_tabentry(dbid, relid);
+ if (tabentry == NULL)
+ /* Table don't exists or isn't an heap relation. */
+ PG_RETURN_NULL();
+
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, tabentry, ncolumns);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
+ Oid storedMyDatabaseId = MyDatabaseId;
+
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+ MyDatabaseId = storedMyDatabaseId;
+
+
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ Oid reloid;
+
+ CHECK_FOR_INTERRUPTS();
+
+ tabentry = (PgStat_StatTabEntry *) entry->data;
+ reloid = entry->key.objoid;
+
+ if (tabentry != NULL)
+ tuplestore_put_for_relation(reloid, tupstore, tupdesc, tabentry, ncolumns);
+ }
+ }
+ PG_RETURN_NULL();
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 943d8588f3d..93db1232df9 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1602,6 +1602,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4abc6d95262..443c4ec65d3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12254,5 +12254,13 @@
proallargtypes => '{int8,pg_lsn,pg_lsn,int4}', proargmodes => '{o,o,o,o}',
proargnames => '{summarized_tli,summarized_lsn,pending_lsn,summarizer_pid}',
prosrc => 'pg_get_wal_summarizer_state' },
-
+{ oid => '8001',
+ descr => 'pg_stat_vacuum_tables return stats values',
+ proname => 'pg_stat_vacuum_tables', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid oid',
+ proallargtypes => '{oid,oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dboid,reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_tables' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f63159c55ca..4492a0572c6 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -167,6 +167,52 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter sync_error_count;
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ int64 total_blks_read; /* number of pages that were missed in shared buffers during a vacuum of specific relation */
+ int64 total_blks_hit; /* number of pages that were found in shared buffers during a vacuum of specific relation */
+ int64 total_blks_dirtied; /* number of pages marked as 'Dirty' during a vacuum of specific relation. */
+ int64 total_blks_written; /* number of pages written during a vacuum of specific relation. */
+
+ int64 blks_fetched; /* number of a relation blocks, fetched during the vacuum. */
+ int64 blks_hit; /* number of a relation blocks, found in shared buffers during the vacuum. */
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double system_time; /* amount of time the CPU was busy executing vacuum code in kernel space, in msec */
+ double user_time; /* amount of time the CPU was busy executing vacuum code in user space, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 interrupts;
+
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -207,6 +253,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -265,7 +321,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAE
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
typedef struct PgStat_ArchiverStats
{
@@ -384,6 +440,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -456,6 +514,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -621,10 +684,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -672,6 +737,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -688,7 +764,9 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
+extern void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
* Functions in pgstat_replslot.c
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 054dd2bf62f..c6225b9cddd 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -228,7 +228,7 @@ extern int err_generic_string(int field, const char *str);
extern int geterrcode(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
-
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index fb132e439dc..715ae1b6fd4 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -549,7 +549,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind, Oid dboid,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, Oid objoid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
@@ -874,4 +874,38 @@ pgstat_get_custom_snapshot_data(PgStat_Kind kind)
return pgStatLocal.snapshot.custom_data[idx];
}
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
#endif /* PGSTAT_INTERNAL_H */
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..7cdb79c0ec4
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 100| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 100| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 6da98cffaca..c612de70083 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..7d31ddbece9
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,51 @@
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
+# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 0d734169f11..9ae743eae0c 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,9 +32,10 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
------+---------
-(0 rows)
+ oid | proname
+------+-----------------------
+ 8001 | pg_stat_vacuum_tables
+(1 row)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 862433ee52b..6e8790f66f6 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2229,6 +2229,40 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stat_vacuum_tables(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..1a7d04b0590
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,200 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | t | t
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+UPDATE vestat SET x = x+1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | f | f
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 2429ec2bbaa..f8a4bcccc9d 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..41e387dd304
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,158 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+UPDATE vestat SET x = x+1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v5-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (41.4K, 4-v5-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 55368c92223b944331c1b18fbbf09e410ac4f478 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 11 Jun 2024 10:11:37 +0300
Subject: [PATCH 2/4] Machinery for grabbing an extended vacuum statistics on
heap and index relations. Remember, statistic on heap and index relations a
bit different (see ExtVacReport to find out more information). The concept of
the ExtVacReport structure has been complicated to store statistic
information for two kinds of relations: for heap and index relations.
ExtVacReportType variable helps to determine what the kind is considering
now.
---
src/backend/access/heap/vacuumlazy.c | 99 +++++++++--
src/backend/catalog/system_views.sql | 41 +++++
src/backend/utils/activity/pgstat.c | 45 +++--
src/backend/utils/activity/pgstat_relation.c | 3 +-
src/backend/utils/adt/pgstatfuncs.c | 108 +++++++-----
src/include/catalog/pg_proc.dat | 9 +
src/include/pgstat.h | 53 ++++--
.../vacuum-extending-in-repetable-read.out | 7 +-
.../vacuum-extending-in-repetable-read.spec | 2 +-
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/expected/rules.out | 26 +++
.../expected/vacuum_index_statistics.out | 158 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 3 +-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 128 ++++++++++++++
15 files changed, 606 insertions(+), 84 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3941ae26f2d..4e2ae78d255 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -168,6 +168,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -246,6 +247,13 @@ typedef struct LVExtStatCounters
PgStat_Counter blocks_hit;
} LVExtStatCounters;
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -408,6 +416,46 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+static void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+static void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -711,14 +759,15 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(rel, &extVacCounters, &extVacReport);
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.pages_frozen = vacrel->set_frozen_pages;
- extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.type = PGSTAT_EXTVAC_HEAP;
+ extVacReport.heap.pages_scanned = vacrel->scanned_pages;
+ extVacReport.heap.pages_removed = vacrel->removed_pages;
+ extVacReport.heap.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.heap.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.heap.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.heap.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.heap.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.heap.index_vacuum_count = vacrel->num_index_scans;
/*
* Report results to the cumulative stats system, too.
@@ -2583,6 +2632,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2601,6 +2654,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2609,6 +2663,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, (void *) vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2633,6 +2694,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2652,12 +2717,20 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3274,7 +3347,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() >= ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3291,7 +3364,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() >= ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3307,16 +3380,22 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 68e6bfe6115..05f8fc07108 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1425,3 +1425,44 @@ WHERE
db.datname = current_database() AND
rel.oid = stats.relid AND
ns.oid = rel.relnamespace;
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_deleted,
+ stats.tuples_deleted,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stat_vacuum_indexes(db.oid, rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace;
+
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 6a788f2b586..75890b18988 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -824,17 +824,33 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->pages_frozen += src->pages_frozen;
- dst->pages_all_visible += src->pages_all_visible;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->dead_tuples += src->dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
}
/* ------------------------------------------------------------
@@ -1081,7 +1097,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
PG_TRY();
{
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
- pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ if (kind == PGSTAT_KIND_RELATION)
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
}
PG_FINALLY();
{
@@ -1136,6 +1153,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index d40d43cdb4a..5b06b04faad 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -211,7 +211,7 @@ pgstat_drop_relation(Relation rel)
* ---------
*/
void
-pgstat_report_vacuum_error(Oid tableoid)
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -228,6 +228,7 @@ pgstat_report_vacuum_error(Oid tableoid)
tabentry = &shtabentry->stats;
tabentry->vacuum_ext.interrupts++;
+ tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index f94e562009d..b868b917ceb 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2035,6 +2035,8 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
}
#define EXTVACHEAPSTAT_COLUMNS 27
+#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
static Oid CurrentDatabaseId = InvalidOid;
@@ -2078,12 +2080,12 @@ static void
tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
TupleDesc tupdesc, PgStat_StatTabEntry *tabentry, int ncolumns)
{
- Datum values[EXTVACHEAPSTAT_COLUMNS];
- bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ Datum values[EXTVACSTAT_COLUMNS];
+ bool nulls[EXTVACSTAT_COLUMNS];
char buf[256];
int i = 0;
- memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+ memset(nulls, 0, EXTVACSTAT_COLUMNS * sizeof(bool));
values[i++] = ObjectIdGetDatum(relid);
@@ -2096,16 +2098,25 @@ tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
tabentry->vacuum_ext.blks_hit);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
- values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
- values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+ if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_HEAP)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ }
+ else if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.pages_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.tuples_deleted);
+ }
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
@@ -2134,7 +2145,7 @@ tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
* Get the vacuum statistics for the heap tables or indexes.
*/
static Datum
-pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
{
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
MemoryContext per_query_ctx;
@@ -2142,7 +2153,6 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Tuplestorestate *tupstore;
TupleDesc tupdesc;
Oid dbid = PG_GETARG_OID(0);
- Oid relid = PG_GETARG_OID(1);
PgStat_StatTabEntry *tabentry;
InitMaterializedSRF(fcinfo, 0);
@@ -2169,40 +2179,45 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
MemoryContextSwitchTo(oldcontext);
- /* Load table statistics for specified database. */
- if (OidIsValid(relid))
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
{
- tabentry = fetch_dbstat_tabentry(dbid, relid);
- if (tabentry == NULL)
- /* Table don't exists or isn't an heap relation. */
- PG_RETURN_NULL();
+ Oid relid = PG_GETARG_OID(1);
- tuplestore_put_for_relation(relid, tupstore, tupdesc, tabentry, ncolumns);
- }
- else
- {
- SnapshotIterator hashiter;
- PgStat_SnapshotEntry *entry;
- Oid storedMyDatabaseId = MyDatabaseId;
+ /* Load table statistics for specified database. */
+ if (OidIsValid(relid))
+ {
+ tabentry = fetch_dbstat_tabentry(dbid, relid);
+ if (tabentry == NULL || tabentry->vacuum_ext.type != type)
+ /* Table don't exists or isn't an heap relation. */
+ PG_RETURN_NULL();
- pgstat_update_snapshot(PGSTAT_KIND_RELATION);
- MyDatabaseId = storedMyDatabaseId;
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, tabentry, ncolumns);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
+ Oid storedMyDatabaseId = MyDatabaseId;
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+ MyDatabaseId = storedMyDatabaseId;
- /* Iterate the snapshot */
- InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
- while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
- {
- Oid reloid;
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ Oid reloid;
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- tabentry = (PgStat_StatTabEntry *) entry->data;
- reloid = entry->key.objoid;
+ tabentry = (PgStat_StatTabEntry *) entry->data;
+ reloid = entry->key.objoid;
- if (tabentry != NULL)
- tuplestore_put_for_relation(reloid, tupstore, tupdesc, tabentry, ncolumns);
+ if (tabentry != NULL && tabentry->vacuum_ext.type == type)
+ tuplestore_put_for_relation(reloid, tupstore, tupdesc, tabentry, ncolumns);
+ }
}
}
PG_RETURN_NULL();
@@ -2214,7 +2229,18 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Datum
pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
{
- return pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
+
+ PG_RETURN_NULL();
+}
+
+/*
+ * Get the vacuum statistics for the indexes.
+ */
+Datum
+pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
PG_RETURN_NULL();
}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 443c4ec65d3..2e80a2502cc 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12263,4 +12263,13 @@
proargmodes => '{i,i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{dboid,reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
prosrc => 'pg_stat_vacuum_tables' },
+{ oid => '8002',
+ descr => 'pg_stat_vacuum_indexes return stats values',
+ proname => 'pg_stat_vacuum_indexes', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid oid',
+ proallargtypes => '{oid,oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dboid,reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_indexes' }
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 4492a0572c6..762b53b88ed 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -167,11 +167,20 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter sync_error_count;
} PgStat_BackendSubEntry;
+
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_HEAP = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -203,14 +212,38 @@ typedef struct ExtVacReport
/* Interruptions on any errors. */
int32 interrupts;
- int64 pages_scanned; /* number of pages we examined */
- int64 pages_removed; /* number of pages removed by vacuum */
- int64 pages_frozen; /* number of pages marked in VM as frozen */
- int64 pages_all_visible; /* number of pages marked in VM as all-visible */
- int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
- int64 index_vacuum_count; /* number of index vacuumings */
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } heap;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -689,7 +722,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 7cdb79c0ec4..93fe15c01f9 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -9,10 +9,9 @@ step s2_print_vacuum_stats_table:
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
-relname |tuples_deleted|dead_tuples|tuples_frozen
---------------------------+--------------+-----------+-------------
-test_vacuum_stat_isolation| 0| 0| 0
-(1 row)
+relname|tuples_deleted|dead_tuples|tuples_frozen
+-------+--------------+-----------+-------------
+(0 rows)
step s1_begin_repeatable_read:
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 7d31ddbece9..bca3e8516b2 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -48,4 +48,4 @@ permutation
s1_commit
s2_checkpoint
s2_vacuum
- s2_print_vacuum_stats_table
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 9ae743eae0c..5d72b970b03 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,10 +32,11 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
-------+-----------------------
+ oid | proname
+------+------------------------
8001 | pg_stat_vacuum_tables
-(1 row)
+ 8002 | pg_stat_vacuum_indexes
+(2 rows)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6e8790f66f6..437aa33f16c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2229,6 +2229,32 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stat_vacuum_indexes(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..a0da8d25f1a
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,158 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+---------+----------+---------------+----------------
+(0 rows)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index 1a7d04b0590..b85a5cab9af 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -37,8 +37,7 @@ FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
---------+--------------+----------------+----------+---------------+---------------
- vestat | 0 | 0 | 455 | 0 | 0
-(1 row)
+(0 rows)
SELECT relpages AS rp
FROM pg_class c
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index f8a4bcccc9d..b9408a43f71 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..9113fd26e6f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,128 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] v5-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (18.9K, 5-v5-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 3eb427caa4b630af6d371b62ee7bf24511eff0f3 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 11 Jun 2024 13:55:39 +0300
Subject: [PATCH 3/4] Machinery for grabbing an extended vacuum statistics on
databases. It transmits vacuum statistical information about each table and
accumulates it for the database which the table belonged.
---
src/backend/catalog/system_views.sql | 28 +++++++
src/backend/utils/activity/pgstat.c | 2 +
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 16 ++++
src/backend/utils/adt/pgstatfuncs.c | 79 +++++++++++++++++++
src/include/catalog/pg_proc.dat | 9 +++
src/include/pgstat.h | 3 +-
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/expected/rules.out | 18 +++++
...ut => vacuum_tables_and_db_statistics.out} | 78 ++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 66 +++++++++++++++-
12 files changed, 303 insertions(+), 6 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (77%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (79%)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 05f8fc07108..ca3ad09727e 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1466,3 +1466,31 @@ WHERE
rel.oid = stats.relid AND
ns.oid = rel.relnamespace;
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+
+ stats.interrupts
+FROM
+ pg_database db LEFT JOIN pg_stat_vacuum_database(db.oid) stats
+ON
+ db.oid = stats.dboid;
+
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 75890b18988..3c50bea379c 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1099,6 +1099,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
if (kind == PGSTAT_KIND_RELATION)
pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ else if (kind == PGSTAT_KIND_DATABASE)
+ pgstat_build_snapshot(PGSTAT_KIND_DATABASE);
}
PG_FINALLY();
{
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 29bc0909748..a060d1a4042 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -430,6 +430,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 5b06b04faad..cc09aba571f 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -217,6 +217,7 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
if (!pgstat_track_counts)
return;
@@ -230,6 +231,10 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
tabentry->vacuum_ext.interrupts++;
tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.interrupts++;
+ dbentry->vacuum_ext.type = m_type;
}
/*
@@ -243,6 +248,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -296,6 +302,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b868b917ceb..0c490ba5f1a 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2036,6 +2036,7 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
#define EXTVACHEAPSTAT_COLUMNS 27
#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACDBSTAT_COLUMNS 15
#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
static Oid CurrentDatabaseId = InvalidOid;
@@ -2076,6 +2077,48 @@ fetch_dbstat_tabentry(Oid dbid, Oid relid)
return tabentry;
}
+static void
+tuplestore_put_for_database(Oid dbid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStatShared_Database *dbentry, int ncolumns)
+{
+ Datum values[EXTVACDBSTAT_COLUMNS];
+ bool nulls[EXTVACDBSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACDBSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->stats.vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(dbentry->stats.vacuum_ext.interrupts);
+
+
+ Assert(i == ncolumns);
+
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
static void
tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
TupleDesc tupdesc, PgStat_StatTabEntry *tabentry, int ncolumns)
@@ -2220,6 +2263,31 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
}
}
}
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ PgStatShared_Database *dbentry;
+ PgStat_EntryRef *entry_ref;
+ Oid storedMyDatabaseId = MyDatabaseId;
+
+ pgstat_update_snapshot(PGSTAT_KIND_DATABASE);
+ MyDatabaseId = storedMyDatabaseId;
+
+ if (OidIsValid(dbid))
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dbid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ if (dbentry == NULL)
+ /* Table doesn't exist or isn't a heap relation */
+ PG_RETURN_NULL();
+
+ tuplestore_put_for_database(dbid, tupstore, tupdesc, dbentry, ncolumns);
+ pgstat_unlock_entry(entry_ref);
+ }
+ else
+ PG_RETURN_NULL();
+ }
PG_RETURN_NULL();
}
@@ -2244,3 +2312,14 @@ pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
PG_RETURN_NULL();
}
+
+/*
+ * Get the vacuum statistics for the database.
+ */
+Datum
+pg_stat_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
+
+ PG_RETURN_NULL();
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2e80a2502cc..b2e881aa89d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12272,4 +12272,13 @@
proargmodes => '{i,i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{dboid,reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
prosrc => 'pg_stat_vacuum_indexes' }
+{ oid => '8003',
+ descr => 'pg_stat_vacuum_database return stats values',
+ proname => 'pg_stat_vacuum_database', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 762b53b88ed..110e9472f3c 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -173,7 +173,8 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_HEAP = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 5d72b970b03..7026de157e4 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,11 +32,12 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
-------+------------------------
+ oid | proname
+------+-------------------------
8001 | pg_stat_vacuum_tables
8002 | pg_stat_vacuum_indexes
-(2 rows)
+ 8003 | pg_stat_vacuum_database
+(3 rows)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 437aa33f16c..c4388dd0da1 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2229,6 +2229,24 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM (pg_database db
+ LEFT JOIN LATERAL pg_stat_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON ((db.oid = stats.dboid)));
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 77%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index b85a5cab9af..f0537aac430 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,6 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE statistic_vacuum_database;
+CREATE DATABASE statistic_vacuum_database1;
+\c statistic_vacuum_database;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -196,4 +199,79 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
t | t | t | t
(1 row)
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+---------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ statistic_vacuum_database | t | t | t | t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c statistic_vacuum_database1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'statistic_vacuum_database';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+---------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ statistic_vacuum_database | t | t | t | t | t | t | t | t
+(1 row)
+
+\c statistic_vacuum_database
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
+\c statistic_vacuum_database1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(d.oid, 0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE statistic_vacuum_database1;
+DROP DATABASE statistic_vacuum_database;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index b9408a43f71..129b1102028 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 79%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 41e387dd304..43cc8068b0f 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,6 +7,10 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE statistic_vacuum_database;
+CREATE DATABASE statistic_vacuum_database1;
+\c statistic_vacuum_database;
+
-- conditio sine qua non
SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
@@ -155,4 +159,64 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+DROP TABLE vestat CASCADE;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c statistic_vacuum_database1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'statistic_vacuum_database';
+
+\c statistic_vacuum_database
+
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
+DROP TABLE vestat CASCADE;
+
+\c statistic_vacuum_database1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(d.oid, 0)
+WHERE oid = 0; -- must be 0
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE statistic_vacuum_database1;
+DROP DATABASE statistic_vacuum_database;
--
2.34.1
[text/x-patch] v5-0004-Add-documentation-about-the-system-views-that-are-us.patch (24.2K, 6-v5-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 7d02ddb8e411b579375ab3466b09862f0d79160a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 15 Aug 2024 11:44:47 +0300
Subject: [PATCH 4/4] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 747 +++++++++++++++++++++++++++++++++
1 file changed, 747 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 634a4c0fab4..42d3ad21486 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5064,4 +5064,751 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stats-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ &project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this index
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ &project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-frozen in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_all_visible</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-visible in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this table
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
+
</chapter>
--
2.34.1
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-15 09:50 Ilia Evdokimov <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 0 replies; 77+ messages in thread
From: Ilia Evdokimov @ 2024-08-15 09:50 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; Andrei Zubkov <[email protected]>; +Cc: pgsql-hackers; [email protected]
On 15.8.24 11:49, Alena Rybakina wrote:
>>
>> I have some suggestions:
>>
>> 1. pgstatfuncs.c in functions tuplestore_put_for_database() and
>> tuplestore_put_for_relation you can remove 'nulls' array if
>> you're sure that columns cannot be NULL.
>>
> We need to use this for tuplestore_putvalues function. With this
> function, we fill the table with the values of the statistics.
Ah, right! I'm sorry.
>> 1.
>>
>>
>>
>> 2. These functions are almost the same and I would think of writing
>> one function depending of type 'ExtVacReportType'
>>
> I'm not sure that I fully understand what you mean. Can you explain it
> more clearly, please?
Ah, I didn't notice that the size of all three tables is different.
Therefore, it won't be possible to write one function instead of two to
avoid code duplication. My mistake.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-16 11:12 jian he <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 2 replies; 77+ messages in thread
From: jian he @ 2024-08-16 11:12 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
On Thu, Aug 15, 2024 at 4:49 PM Alena Rybakina
<[email protected]> wrote:
>
> Hi!
I've applied all the v5 patches.
0002 and 0003 have white space errors.
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was
not necessary
+ (this only includes hits in the
+ &project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was
not necessary
+ (this only includes hits in the
+ &project; buffer cache, not the operating system's file system cache)
+ </para></entry>
"&project;"
represents a sgml file placeholder name as "project" and puts all the
content of "project.sgml" to system-views.sgml.
but you don't have "project.sgml". you may check
doc/src/sgml/filelist.sgml or doc/src/sgml/ref/allfiles.sgml
for usage of "&place_holder;".
so you can change it to "project", otherwise doc cannot build.
src/backend/commands/dbcommands.c
we have:
/*
* If built with appropriate switch, whine when regression-testing
* conventions for database names are violated. But don't complain during
* initdb.
*/
#ifdef ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
if (IsUnderPostmaster && strstr(dbname, "regression") == NULL)
elog(WARNING, "databases created by regression test cases
should have names including \"regression\"");
#endif
so in src/test/regress/sql/vacuum_tables_and_db_statistics.sql you
need to change dbname:
CREATE DATABASE statistic_vacuum_database;
CREATE DATABASE statistic_vacuum_database1;
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
TOAST should
<acronym>TOAST</acronym>
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
maybe change to
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("return type must be a row type")));
Later I found out "InitMaterializedSRF(fcinfo, 0);" already did all
the work. much of the code can be gotten rid of.
please check attached.
>>>>
#define EXTVACHEAPSTAT_COLUMNS 27
#define EXTVACIDXSTAT_COLUMNS 19
#define EXTVACDBSTAT_COLUMNS 15
#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
static Oid CurrentDatabaseId = InvalidOid;
>>>>
we already defined MyDatabaseId in src/include/miscadmin.h,
Why do we need "static Oid CurrentDatabaseId = InvalidOid;"?
also src/backend/utils/adt/pgstatfuncs.c already included "miscadmin.h".
the following code one function has 2 return statements?
------------------------------------------------------------------------
/*
* Get the vacuum statistics for the heap tables.
*/
Datum
pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
{
return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
PG_RETURN_NULL();
}
/*
* Get the vacuum statistics for the indexes.
*/
Datum
pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
{
return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
PG_RETURN_NULL();
}
/*
* Get the vacuum statistics for the database.
*/
Datum
pg_stat_vacuum_database(PG_FUNCTION_ARGS)
{
return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
PG_RETURN_NULL();
}
------------------------------------------------------------------------
in pg_stats_vacuum:
if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
{
Oid relid = PG_GETARG_OID(1);
/* Load table statistics for specified database. */
if (OidIsValid(relid))
{
tabentry = fetch_dbstat_tabentry(dbid, relid);
if (tabentry == NULL || tabentry->vacuum_ext.type != type)
/* Table don't exists or isn't an heap relation. */
PG_RETURN_NULL();
tuplestore_put_for_relation(relid, tupstore, tupdesc,
tabentry, ncolumns);
}
else
{
...
}
}
I don't understand the ELSE branch. the IF branch means the input
dboid, heap/index oid is correct.
the ELSE branch means table reloid is invalid = 0.
I am not sure 100% what the ELSE Branch means.
Also there are no comments explaining why.
experiments seem to show that when reloid is 0, it will print out all
the vacuum statistics
for all the tables in the current database. If so, then only super
users can call pg_stats_vacuum?
but the table owner should be able to call pg_stats_vacuum for that
specific table.
/* Type of ExtVacReport */
typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_HEAP = 1,
PGSTAT_EXTVAC_INDEX = 2,
PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
generally "HEAP" means table and index, maybe "PGSTAT_EXTVAC_HEAP" would be term
Attachments:
[application/octet-stream] v5-0001-minor-refactor-pg_stats_vacuum-and-sub-routine.no-cfbot (4.5K, 2-v5-0001-minor-refactor-pg_stats_vacuum-and-sub-routine.no-cfbot)
download
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-19 09:32 jian he <[email protected]>
parent: jian he <[email protected]>
1 sibling, 2 replies; 77+ messages in thread
From: jian he @ 2024-08-19 09:32 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
in pg_stats_vacuum
if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
{
Oid relid = PG_GETARG_OID(1);
/* Load table statistics for specified database. */
if (OidIsValid(relid))
{
tabentry = fetch_dbstat_tabentry(dbid, relid);
if (tabentry == NULL || tabentry->vacuum_ext.type != type)
/* Table don't exists or isn't an heap relation. */
PG_RETURN_NULL();
tuplestore_put_for_relation(relid, rsinfo, tabentry);
}
else
{
}
So for functions pg_stat_vacuum_indexes and pg_stat_vacuum_tables,
it seems you didn't check "relid" 's relkind,
you may need to use get_rel_relkind.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-19 16:28 Ilia Evdokimov <[email protected]>
parent: jian he <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Ilia Evdokimov @ 2024-08-19 16:28 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]; jian he <[email protected]>
Are you certain that all tables are included in `pg_stat_vacuum_tables`?
I'm asking because of the following:
SELECT count(*) FROM pg_stat_all_tables ;
count
-------
108
(1 row)
SELECT count(*) FROM pg_stat_vacuum_tables ;
count
-------
20
(1 row)
--
Regards,
Ilia Evdokimov,
Tantor Labs LCC.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-20 22:35 Alena Rybakina <[email protected]>
parent: jian he <[email protected]>
1 sibling, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2024-08-20 22:35 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
Hi! Thank you very much for your review! Sorry for my late response I
was overwhelmed by tasks.
On 16.08.2024 14:12, jian he wrote:
> On Thu, Aug 15, 2024 at 4:49 PM Alena Rybakina
> <[email protected]> wrote:
>> Hi!
>
> I've applied all the v5 patches.
> 0002 and 0003 have white space errors.
>
> + <para>
> + Number of times blocks of this index were already found
> + in the buffer cache by vacuum operations, so that a read was
> not necessary
> + (this only includes hits in the
> + &project; buffer cache, not the operating system's file system cache)
> + </para></entry>
>
> + Number of times blocks of this table were already found
> + in the buffer cache by vacuum operations, so that a read was
> not necessary
> + (this only includes hits in the
> + &project; buffer cache, not the operating system's file system cache)
> + </para></entry>
>
> "&project;"
> represents a sgml file placeholder name as "project" and puts all the
> content of "project.sgml" to system-views.sgml.
> but you don't have "project.sgml". you may check
> doc/src/sgml/filelist.sgml or doc/src/sgml/ref/allfiles.sgml
> for usage of "&place_holder;".
> so you can change it to "project", otherwise doc cannot build.
>
>
> src/backend/commands/dbcommands.c
> we have:
> /*
> * If built with appropriate switch, whine when regression-testing
> * conventions for database names are violated. But don't complain during
> * initdb.
> */
> #ifdef ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
> if (IsUnderPostmaster && strstr(dbname, "regression") == NULL)
> elog(WARNING, "databases created by regression test cases
> should have names including \"regression\"");
> #endif
> so in src/test/regress/sql/vacuum_tables_and_db_statistics.sql you
> need to change dbname:
> CREATE DATABASE statistic_vacuum_database;
> CREATE DATABASE statistic_vacuum_database1;
>
>
> + <para>
> + The view <structname>pg_stat_vacuum_indexes</structname> will contain
> + one row for each index in the current database (including TOAST
> + table indexes), showing statistics about vacuuming that specific index.
> + </para>
> TOAST should
> <acronym>TOAST</acronym>
>
>
>
> + /* Build a tuple descriptor for our result type */
> + if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
> + elog(ERROR, "return type must be a row type");
> maybe change to
> ereport(ERROR,
> (errcode(ERRCODE_DATATYPE_MISMATCH),
> errmsg("return type must be a row type")));
> Later I found out "InitMaterializedSRF(fcinfo, 0);" already did all
> the work. much of the code can be gotten rid of.
> please check attached.
I agree with your suggestions for improving the code. I will add this in
the next version of the patch.
>
> #define EXTVACHEAPSTAT_COLUMNS 27
> #define EXTVACIDXSTAT_COLUMNS 19
> #define EXTVACDBSTAT_COLUMNS 15
> #define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
>
> static Oid CurrentDatabaseId = InvalidOid;
> we already defined MyDatabaseId in src/include/miscadmin.h,
> Why do we need "static Oid CurrentDatabaseId = InvalidOid;"?
> also src/backend/utils/adt/pgstatfuncs.c already included "miscadmin.h".
Hmm, Tom Lane added "misc admin.h", or I didn't notice something. Could
you point this out, please?
We used the Current Database Id to output statistics on tables from
another database, so we need to replace it with a different default
value. But I want to rewrite this patch to display table statistics only
for the current database, that is, this part will be removed in the
future. In my opinion, it would be more correct.
> the following code one function has 2 return statements?
> ------------------------------------------------------------------------
> /*
> * Get the vacuum statistics for the heap tables.
> */
> Datum
> pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
> {
> return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
>
> PG_RETURN_NULL();
> }
>
> /*
> * Get the vacuum statistics for the indexes.
> */
> Datum
> pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
> {
> return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
>
> PG_RETURN_NULL();
> }
>
> /*
> * Get the vacuum statistics for the database.
> */
> Datum
> pg_stat_vacuum_database(PG_FUNCTION_ARGS)
> {
> return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
>
> PG_RETURN_NULL();
> }
You are right - the second return is superfluous. I'll fix it.
> ------------------------------------------------------------------------
> in pg_stats_vacuum:
> if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
> {
> Oid relid = PG_GETARG_OID(1);
>
> /* Load table statistics for specified database. */
> if (OidIsValid(relid))
> {
> tabentry = fetch_dbstat_tabentry(dbid, relid);
> if (tabentry == NULL || tabentry->vacuum_ext.type != type)
> /* Table don't exists or isn't an heap relation. */
> PG_RETURN_NULL();
>
> tuplestore_put_for_relation(relid, tupstore, tupdesc,
> tabentry, ncolumns);
> }
> else
> {
> ...
> }
> }
> I don't understand the ELSE branch. the IF branch means the input
> dboid, heap/index oid is correct.
> the ELSE branch means table reloid is invalid = 0.
> I am not sure 100% what the ELSE Branch means.
> Also there are no comments explaining why.
> experiments seem to show that when reloid is 0, it will print out all
> the vacuum statistics
> for all the tables in the current database. If so, then only super
> users can call pg_stats_vacuum?
> but the table owner should be able to call pg_stats_vacuum for that
> specific table.
If any reloid has not been set by the user, we output statistics for all
objects - tables or indexes.In this part of the code, we find all the
suitable objects from the snapshot, if they belong to the index or table
type of objects.
> /* Type of ExtVacReport */
> typedef enum ExtVacReportType
> {
> PGSTAT_EXTVAC_INVALID = 0,
> PGSTAT_EXTVAC_HEAP = 1,
> PGSTAT_EXTVAC_INDEX = 2,
> PGSTAT_EXTVAC_DB = 3,
> } ExtVacReportType;
> generally "HEAP" means table and index, maybe "PGSTAT_EXTVAC_HEAP" would be term
No, Heap means something like a table in a relationship database, or its
alternative name is Heap.
--
Regards,
Alena Rybakina
Postgres Professional:http://www.postgrespro.com
The Russian Postgres Company
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-20 22:37 Alena Rybakina <[email protected]>
parent: jian he <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2024-08-20 22:37 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
We check it there: "tabentry->vacuum_ext.type != type". Or were you
talking about something else?
On 19.08.2024 12:32, jian he wrote:
> in pg_stats_vacuum
> if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
> {
> Oid relid = PG_GETARG_OID(1);
>
> /* Load table statistics for specified database. */
> if (OidIsValid(relid))
> {
> tabentry = fetch_dbstat_tabentry(dbid, relid);
> if (tabentry == NULL || tabentry->vacuum_ext.type != type)
> /* Table don't exists or isn't an heap relation. */
> PG_RETURN_NULL();
>
> tuplestore_put_for_relation(relid, rsinfo, tabentry);
> }
> else
> {
> }
>
>
> So for functions pg_stat_vacuum_indexes and pg_stat_vacuum_tables,
> it seems you didn't check "relid" 's relkind,
> you may need to use get_rel_relkind.
--
Regards,
Alena Rybakina
Postgres Professional:http://www.postgrespro.com
The Russian Postgres Company
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-20 22:39 Alena Rybakina <[email protected]>
parent: Ilia Evdokimov <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2024-08-20 22:39 UTC (permalink / raw)
To: Ilia Evdokimov <[email protected]>; +Cc: Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]; jian he <[email protected]>
I think you've counted the above system tables from the database, but
I'll double-check it. Thank you for your review!
On 19.08.2024 19:28, Ilia Evdokimov wrote:
> Are you certain that all tables are included in
> `pg_stat_vacuum_tables`? I'm asking because of the following:
>
>
> SELECT count(*) FROM pg_stat_all_tables ;
> count
> -------
> 108
> (1 row)
>
> SELECT count(*) FROM pg_stat_vacuum_tables ;
> count
> -------
> 20
> (1 row)
>
--
Regards,
Alena Rybakina
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-22 02:47 jian he <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 2 replies; 77+ messages in thread
From: jian he @ 2024-08-22 02:47 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
On Wed, Aug 21, 2024 at 6:37 AM Alena Rybakina
<[email protected]> wrote:
>
> We check it there: "tabentry->vacuum_ext.type != type". Or were you talking about something else?
>
> On 19.08.2024 12:32, jian he wrote:
>
> in pg_stats_vacuum
> if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
> {
> Oid relid = PG_GETARG_OID(1);
>
> /* Load table statistics for specified database. */
> if (OidIsValid(relid))
> {
> tabentry = fetch_dbstat_tabentry(dbid, relid);
> if (tabentry == NULL || tabentry->vacuum_ext.type != type)
> /* Table don't exists or isn't an heap relation. */
> PG_RETURN_NULL();
>
> tuplestore_put_for_relation(relid, rsinfo, tabentry);
> }
> else
> {
> }
>
>
> So for functions pg_stat_vacuum_indexes and pg_stat_vacuum_tables,
> it seems you didn't check "relid" 's relkind,
> you may need to use get_rel_relkind.
>
> --
hi.
I mentioned some points at [1],
Please check the attached patchset to address these issues.
there are four occurrences of "CurrentDatabaseId", i am still confused
with usage of CurrentDatabaseId.
also please don't top-post, otherwise the archive, like [2] is not
easier to read for future readers.
generally you quote first, then reply.
[1] https://postgr.es/m/CACJufxHb_YGCp=pVH6DZcpk9yML+SueffPeaRbX2LzXZVahd_w@mail.gmail.com
[2] https://postgr.es/m/[email protected]
Attachments:
[application/octet-stream] v6-0002-minor-doc-change-to-make-build-successfuly.no-cfbot (1.5K, 2-v6-0002-minor-doc-change-to-make-build-successfuly.no-cfbot)
download
[application/octet-stream] v6-0001-minor-refactor-pg_stats_vacuum-and-sub-routine.no-cfbot (4.5K, 3-v6-0001-minor-refactor-pg_stats_vacuum-and-sub-routine.no-cfbot)
download
[application/octet-stream] v6-0004-ensure-pg_stats_vacuum-object-is-either-relati.no-cfbot (2.1K, 4-v6-0004-ensure-pg_stats_vacuum-object-is-either-relati.no-cfbot)
download
[application/octet-stream] v6-0003-refactor-regression-test.no-cfbot (5.9K, 5-v6-0003-refactor-regression-test.no-cfbot)
download
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-22 04:29 Kirill Reshke <[email protected]>
parent: jian he <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Kirill Reshke @ 2024-08-22 04:29 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: Alena Rybakina <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
On Thu, 22 Aug 2024 at 07:48, jian he <[email protected]> wrote:
>
> On Wed, Aug 21, 2024 at 6:37 AM Alena Rybakina
> <[email protected]> wrote:
> >
> > We check it there: "tabentry->vacuum_ext.type != type". Or were you talking about something else?
> >
> > On 19.08.2024 12:32, jian he wrote:
> >
> > in pg_stats_vacuum
> > if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
> > {
> > Oid relid = PG_GETARG_OID(1);
> >
> > /* Load table statistics for specified database. */
> > if (OidIsValid(relid))
> > {
> > tabentry = fetch_dbstat_tabentry(dbid, relid);
> > if (tabentry == NULL || tabentry->vacuum_ext.type != type)
> > /* Table don't exists or isn't an heap relation. */
> > PG_RETURN_NULL();
> >
> > tuplestore_put_for_relation(relid, rsinfo, tabentry);
> > }
> > else
> > {
> > }
> >
> >
> > So for functions pg_stat_vacuum_indexes and pg_stat_vacuum_tables,
> > it seems you didn't check "relid" 's relkind,
> > you may need to use get_rel_relkind.
> >
> > --
>
> hi.
> I mentioned some points at [1],
> Please check the attached patchset to address these issues.
>
> there are four occurrences of "CurrentDatabaseId", i am still confused
> with usage of CurrentDatabaseId.
>
> also please don't top-post, otherwise the archive, like [2] is not
> easier to read for future readers.
> generally you quote first, then reply.
>
> [1] https://postgr.es/m/CACJufxHb_YGCp=pVH6DZcpk9yML+SueffPeaRbX2LzXZVahd_w@mail.gmail.com
> [2] https://postgr.es/m/[email protected]
Hi, your points are valid.
Regarding 0003, I also wanted to object database naming in a
regression test during my review but for some reason didn't.Now, as
soon as we already need to change it, I suggest we also change
regression_statistic_vacuum_db1 to something less generic. Maybe
regression_statistic_vacuum_db_unaffected.
--
Best regards,
Kirill Reshke
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-23 01:07 Alexander Korotkov <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alexander Korotkov @ 2024-08-23 01:07 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]; jian he <[email protected]>
On Wed, Aug 21, 2024 at 1:39 AM Alena Rybakina <[email protected]>
wrote:
>
> I think you've counted the above system tables from the database, but
> I'll double-check it. Thank you for your review!
>
> On 19.08.2024 19:28, Ilia Evdokimov wrote:
> > Are you certain that all tables are included in
> > `pg_stat_vacuum_tables`? I'm asking because of the following:
> >
> >
> > SELECT count(*) FROM pg_stat_all_tables ;
> > count
> > -------
> > 108
> > (1 row)
> >
> > SELECT count(*) FROM pg_stat_vacuum_tables ;
> > count
> > -------
> > 20
> > (1 row)
> >
I'd like to do some review a well.
+ MyDatabaseId = dbid;
+
+ PG_TRY();
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ MyDatabaseId = storedMyDatabaseId;
+ }
+ PG_CATCH();
+ {
+ MyDatabaseId = storedMyDatabaseId;
+ }
+ PG_END_TRY();
I think this is generally wrong to change MyDatabaseId, especially if you
have to wrap it with PG_TRY()/PG_CATCH(). I think, instead we need proper
API changes, i.e. make pgstat_fetch_stat_tabentry() and others take dboid
as an argument.
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP,
EXTVACHEAPSTAT_COLUMNS);
+
+ PG_RETURN_NULL();
+}
The PG_RETURN_NULL() is unneeded after another return statement. However,
does pg_stats_vacuum() need to return anything? What about making its
return type void?
@@ -874,4 +874,38 @@ pgstat_get_custom_snapshot_data(PgStat_Kind kind)
return pgStatLocal.snapshot.custom_data[idx];
}
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
It would be nice to preserve encapsulation and don't expose pgstat_snapshot
hash in the headers. I see there is only one usage of it outside of
pgstat.c: pg_stats_vacuum().
+ Oid storedMyDatabaseId = MyDatabaseId;
+
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+ MyDatabaseId = storedMyDatabaseId;
This manipulation with storedMyDatabaseId looks pretty useless. It seems
to be intended to change MyDatabaseId, while I'm not fan of this as I
mentioned above.
+static PgStat_StatTabEntry *
+fetch_dbstat_tabentry(Oid dbid, Oid relid)
+{
+ Oid storedMyDatabaseId = MyDatabaseId;
+ PgStat_StatTabEntry *tabentry = NULL;
+
+ if (OidIsValid(CurrentDatabaseId) && CurrentDatabaseId == dbid)
+ /* Quick path when we read data from the same database */
+ return pgstat_fetch_stat_tabentry(relid);
+
+ pgstat_clear_snapshot();
It looks scary to reset the whole snapshot each time we access another
database. Need to also mention that the CurrentDatabaseId machinery isn't
implemented.
New functions
pg_stat_vacuum_tables(), pg_stat_vacuum_indexes(), pg_stat_vacuum_database()
are SRFs. When zero Oid is passed they report all the objects. However,
it seems they aren't intended to be used directly. Instead, there are
views with the same names. These views always call them with particular
Oids, therefore SRFs always return one row. Then why bother with SRF?
They could return plain records instead.
Also, as I mentioned above patchset makes a lot of trouble accessing
statistics of relations of another database. But that seems to be useless
given corresponding views allow to see only relations of the current
database. Even if you call functions directly, what is the value of this
information given that you don't know the relation oids in another
database? So, I think if we will give up and limit access to the relations
of the current database patch will become simpler and clearer.
------
Regards,
Alexander Korotkov
Supabase
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-25 15:59 Alena Rybakina <[email protected]>
parent: Alexander Korotkov <[email protected]>
0 siblings, 3 replies; 77+ messages in thread
From: Alena Rybakina @ 2024-08-25 15:59 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]; jian he <[email protected]>
Hi!
On 23.08.2024 04:07, Alexander Korotkov wrote:
> On Wed, Aug 21, 2024 at 1:39 AM Alena Rybakina
> <[email protected]> wrote:
> >
> > I think you've counted the above system tables from the database, but
> > I'll double-check it. Thank you for your review!
> >
> > On 19.08.2024 19:28, Ilia Evdokimov wrote:
> > > Are you certain that all tables are included in
> > > `pg_stat_vacuum_tables`? I'm asking because of the following:
> > >
> > >
> > > SELECT count(*) FROM pg_stat_all_tables ;
> > > count
> > > -------
> > > 108
> > > (1 row)
> > >
> > > SELECT count(*) FROM pg_stat_vacuum_tables ;
> > > count
> > > -------
> > > 20
> > > (1 row)
> > >
>
> I'd like to do some review a well.
Thank you very much for your review and contribution to this thread!
>
> + MyDatabaseId = dbid;
> +
> + PG_TRY();
> + {
> + tabentry = pgstat_fetch_stat_tabentry(relid);
> + MyDatabaseId = storedMyDatabaseId;
> + }
> + PG_CATCH();
> + {
> + MyDatabaseId = storedMyDatabaseId;
> + }
> + PG_END_TRY();
>
> I think this is generally wrong to change MyDatabaseId, especially if
> you have to wrap it with PG_TRY()/PG_CATCH(). I think, instead we
> need proper API changes, i.e. make pgstat_fetch_stat_tabentry() and
> others take dboid as an argument.
I fixed it by deleting this part pf the code. We can display statistics
only for current database.
>
> +/*
> + * Get the vacuum statistics for the heap tables.
> + */
> +Datum
> +pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
> +{
> + return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP,
> EXTVACHEAPSTAT_COLUMNS);
> +
> + PG_RETURN_NULL();
> +}
>
> The PG_RETURN_NULL() is unneeded after another return statement.
> However, does pg_stats_vacuum() need to return anything? What about
> making its return type void?
I think you are right, we can not return anything. Fixed.
>
> @@ -874,4 +874,38 @@ pgstat_get_custom_snapshot_data(PgStat_Kind kind)
> return pgStatLocal.snapshot.custom_data[idx];
> }
>
> +/* hash table for statistics snapshots entry */
> +typedef struct PgStat_SnapshotEntry
> +{
> + PgStat_HashKey key;
> + char status; /* for simplehash use */
> + void *data; /* the stats data itself */
> +} PgStat_SnapshotEntry;
>
> It would be nice to preserve encapsulation and don't
> expose pgstat_snapshot hash in the headers. I see there is only one
> usage of it outside of pgstat.c: pg_stats_vacuum().
Fixed.
>
> + Oid storedMyDatabaseId = MyDatabaseId;
> +
> + pgstat_update_snapshot(PGSTAT_KIND_RELATION);
> + MyDatabaseId = storedMyDatabaseId;
>
> This manipulation with storedMyDatabaseId looks pretty useless. It
> seems to be intended to change MyDatabaseId, while I'm not fan of this
> as I mentioned above.
Fixed.
>
> +static PgStat_StatTabEntry *
> +fetch_dbstat_tabentry(Oid dbid, Oid relid)
> +{
> + Oid storedMyDatabaseId = MyDatabaseId;
> + PgStat_StatTabEntry *tabentry = NULL;
> +
> + if (OidIsValid(CurrentDatabaseId) && CurrentDatabaseId == dbid)
> + /* Quick path when we read data from the same database */
> + return pgstat_fetch_stat_tabentry(relid);
> +
> + pgstat_clear_snapshot();
>
> It looks scary to reset the whole snapshot each time we access another
> database. Need to also mention that the CurrentDatabaseId machinery
> isn't implemented.
Fixed.
>
> New functions
> pg_stat_vacuum_tables(), pg_stat_vacuum_indexes(), pg_stat_vacuum_database()
> are SRFs. When zero Oid is passed they report all the objects.
> However, it seems they aren't intended to be used directly. Instead,
> there are views with the same names. These views always call them with
> particular Oids, therefore SRFs always return one row. Then why
> bother with SRF? They could return plain records instead.
I didn't understand correctly - did you mean that we don't need SRF if
we need to display statistics for a specific object?
Otherwise, we need this when we display information on all database
objects (tables or indexes):
while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter))
!= NULL)
{
CHECK_FOR_INTERRUPTS();
tabentry = (PgStat_StatTabEntry *) entry->data;
if (tabentry != NULL && tabentry->vacuum_ext.type == type)
tuplestore_put_for_relation(relid, rsinfo, tabentry);
}
I know we can construct a HeapTuple object containing a TupleDesc,
values, and nulls for a particular object, but I'm not sure we can
augment it while looping through multiple objects.
/* Initialise attributes information in the tuple descriptor */
tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_STATS_COLS);
...
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
If I missed something or misunderstood, can you explain in more detail?
>
> Also, as I mentioned above patchset makes a lot of trouble accessing
> statistics of relations of another database. But that seems to be
> useless given corresponding views allow to see only relations of the
> current database. Even if you call functions directly, what is the
> value of this information given that you don't know the relation oids
> in another database? So, I think if we will give up and limit access
> to the relations of the current database patch will become simpler and
> clearer.
>
I agree with that and have fixed it already.
--
Regards,
Alena Rybakina
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Attachments:
[text/x-patch] v6-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (63.5K, 2-v6-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 8903b692430e0e999665bc4a41d5fd088749131e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 10:49:40 +0300
Subject: [PATCH 1/4] Machinery for grabbing an extended vacuum statistics on
heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 159 +++++++++++++-
src/backend/access/heap/visibilitymap.c | 13 ++
src/backend/catalog/system_views.sql | 54 +++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 65 +++++-
src/backend/utils/activity/pgstat_relation.c | 35 ++-
src/backend/utils/adt/pgstatfuncs.c | 157 ++++++++++++++
src/backend/utils/error/elog.c | 13 ++
src/include/catalog/pg_proc.dat | 10 +-
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 84 +++++++-
src/include/utils/elog.h | 1 +
src/include/utils/pgstat_internal.h | 2 +-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 51 +++++
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/expected/rules.out | 34 +++
.../expected/vacuum_tables_statistics.out | 200 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 158 ++++++++++++++
22 files changed, 1092 insertions(+), 16 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d82aa3d4896..3941ae26f2d 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -167,6 +167,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -194,6 +195,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -226,6 +229,22 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Cut-off values of parameters which changes implicitly during a vacuum
+ * process.
+ * Vacuum can't control their values, so we should store them before and after
+ * the processing.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz time;
+ PGRUsage ru;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -279,6 +298,115 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+ PGRUsage ru0;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ pg_rusage_init(&ru0);
+ starttime = GetCurrentTimestamp();
+
+ counters->ru = ru0;
+ counters->time = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+ PGRUsage ru1;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->time, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ /*
+ * Get difference of a system time and user time values in milliseconds.
+ * Use floating point representation to show tails of time diffs.
+ */
+ pg_rusage_init(&ru1);
+ report->system_time =
+ (ru1.ru.ru_stime.tv_sec - counters->ru.ru.ru_stime.tv_sec) * 1000. +
+ (ru1.ru.ru_stime.tv_usec - counters->ru.ru.ru_stime.tv_usec) * 0.001;
+ report->user_time =
+ (ru1.ru.ru_utime.tv_sec - counters->ru.ru.ru_utime.tv_sec) * 1000. +
+ (ru1.ru.ru_utime.tv_usec - counters->ru.ru.ru_utime.tv_usec) * 0.001;
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -311,6 +439,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -329,7 +459,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -346,6 +476,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -413,6 +544,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -574,6 +707,19 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -588,7 +734,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1380,6 +1527,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2277,11 +2426,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -3122,6 +3273,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3137,6 +3290,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33c..d72cade60a4 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,18 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Initially, it didn't matter what type of flags (all-visible or frozen) we received,
+ * we just performed a reverse concatenation operation. But this information is very important
+ * for vacuum statistics. We need to find out this usingthe bit concatenation operation
+ * with the VISIBILITYMAP_ALL_VISIBLE and VISIBILITYMAP_ALL_FROZEN masks,
+ * and where the desired one matches, we increment the value there.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 19cabc9a47f..e84d6881403 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1371,3 +1371,57 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stat_vacuum_tables(rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace;
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 7d8e9d20454..363924d00db 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -103,6 +103,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2394,6 +2397,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 22c057fe61b..13ab633086a 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1043,6 +1043,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index b2ca3f39b7a..4e8d8f8dc77 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -260,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -830,6 +829,40 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
+void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->system_time += src->system_time;
+ dst->user_time += src->user_time;
+ dst->total_time += src->total_time;
+ dst->interrupts += src->interrupts;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->dead_tuples += src->dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+}
/* ------------------------------------------------------------
* Fetching of stats
@@ -896,7 +929,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, Oid objoid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1012,7 +1045,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1062,8 +1095,30 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 8a3f7d434cf..d40d43cdb4a 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -204,12 +204,40 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.interrupts++;
+ pgstat_unlock_entry(entry_ref);
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -233,6 +261,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -861,6 +891,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 32211371237..1df271286e6 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -31,6 +31,42 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+#include "utils/pgstat_internal.h"
+
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -2032,3 +2068,124 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objoid));
}
+
+#define EXTVACHEAPSTAT_COLUMNS 27
+
+static void
+tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
+ PgStat_StatTabEntry *tabentry)
+{
+ Datum values[EXTVACHEAPSTAT_COLUMNS];
+ bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_fetched -
+ tabentry->vacuum_ext.blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, tabentry->vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(tabentry->vacuum_ext.interrupts);
+
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+}
+
+/*
+ * Get the vacuum statistics for the heap tables or indexes.
+ */
+static void
+pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* Check if caller supports us returning a tuplestore */
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("set-valued function called in context that cannot accept a set")));
+ Assert(rsinfo->setDesc->natts == ncolumns);
+ Assert(rsinfo->setResult != NULL);
+
+ /* Load table statistics for specified database. */
+ if (OidIsValid(relid))
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ if (tabentry == NULL)
+ /* Table don't exists or isn't an heap relation. */
+ return;
+
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
+
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ Oid reloid;
+
+ CHECK_FOR_INTERRUPTS();
+
+ tabentry = (PgStat_StatTabEntry *) entry->data;
+ reloid = entry->key.objoid;
+
+ if (tabentry != NULL)
+ tuplestore_put_for_relation(reloid, rsinfo, tabentry);
+ }
+ }
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 5cbb5b54168..5ead2a8aff8 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4abc6d95262..2023270f923 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12254,5 +12254,13 @@
proallargtypes => '{int8,pg_lsn,pg_lsn,int4}', proargmodes => '{o,o,o,o}',
proargnames => '{summarized_tli,summarized_lsn,pending_lsn,summarizer_pid}',
prosrc => 'pg_get_wal_summarizer_state' },
-
+{ oid => '8001',
+ descr => 'pg_stat_vacuum_tables return stats values',
+ proname => 'pg_stat_vacuum_tables', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_tables' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f63159c55ca..4492a0572c6 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -167,6 +167,52 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter sync_error_count;
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ int64 total_blks_read; /* number of pages that were missed in shared buffers during a vacuum of specific relation */
+ int64 total_blks_hit; /* number of pages that were found in shared buffers during a vacuum of specific relation */
+ int64 total_blks_dirtied; /* number of pages marked as 'Dirty' during a vacuum of specific relation. */
+ int64 total_blks_written; /* number of pages written during a vacuum of specific relation. */
+
+ int64 blks_fetched; /* number of a relation blocks, fetched during the vacuum. */
+ int64 blks_hit; /* number of a relation blocks, found in shared buffers during the vacuum. */
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double system_time; /* amount of time the CPU was busy executing vacuum code in kernel space, in msec */
+ double user_time; /* amount of time the CPU was busy executing vacuum code in user space, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 interrupts;
+
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -207,6 +253,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -265,7 +321,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAE
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
typedef struct PgStat_ArchiverStats
{
@@ -384,6 +440,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -456,6 +514,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -621,10 +684,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -672,6 +737,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -688,7 +764,9 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
+extern void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
* Functions in pgstat_replslot.c
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index e54eca5b489..e752c0ce015 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index fb132e439dc..24ab3ceb717 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -549,7 +549,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind, Oid dboid,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, Oid objoid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..7cdb79c0ec4
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 100| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 100| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 6da98cffaca..c612de70083 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..7d31ddbece9
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,51 @@
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
+# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 0d734169f11..9ae743eae0c 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,9 +32,10 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
------+---------
-(0 rows)
+ oid | proname
+------+-----------------------
+ 8001 | pg_stat_vacuum_tables
+(1 row)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 862433ee52b..cc0b5bde0a1 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2229,6 +2229,40 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..1a7d04b0590
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,200 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | t | t
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+UPDATE vestat SET x = x+1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | f | f
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 2429ec2bbaa..f8a4bcccc9d 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..41e387dd304
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,158 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+UPDATE vestat SET x = x+1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v6-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (40.7K, 3-v6-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 35aac7704eabeaefd5ab76bb18c0e68d29388be1 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:09:21 +0300
Subject: [PATCH 2/4] Machinery for grabbing an extended vacuum statistics on
heap and index relations. Remember, statistic on heap and index relations a
bit different (see ExtVacReport to find out more information). The concept of
the ExtVacReport structure has been complicated to store statistic
information for two kinds of relations: for heap and index relations.
ExtVacReportType variable helps to determine what the kind is considering
now.
---
src/backend/access/heap/vacuumlazy.c | 99 +++++++++--
src/backend/catalog/system_views.sql | 41 +++++
src/backend/utils/activity/pgstat.c | 45 +++--
src/backend/utils/activity/pgstat_relation.c | 3 +-
src/backend/utils/adt/pgstatfuncs.c | 99 ++++++-----
src/include/catalog/pg_proc.dat | 9 +
src/include/pgstat.h | 53 ++++--
.../vacuum-extending-in-repetable-read.out | 7 +-
.../vacuum-extending-in-repetable-read.spec | 2 +-
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/expected/rules.out | 26 +++
.../expected/vacuum_index_statistics.out | 158 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 3 +-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 128 ++++++++++++++
15 files changed, 600 insertions(+), 81 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3941ae26f2d..4e2ae78d255 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -168,6 +168,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -246,6 +247,13 @@ typedef struct LVExtStatCounters
PgStat_Counter blocks_hit;
} LVExtStatCounters;
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -408,6 +416,46 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+static void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+static void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -711,14 +759,15 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(rel, &extVacCounters, &extVacReport);
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.pages_frozen = vacrel->set_frozen_pages;
- extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.type = PGSTAT_EXTVAC_HEAP;
+ extVacReport.heap.pages_scanned = vacrel->scanned_pages;
+ extVacReport.heap.pages_removed = vacrel->removed_pages;
+ extVacReport.heap.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.heap.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.heap.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.heap.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.heap.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.heap.index_vacuum_count = vacrel->num_index_scans;
/*
* Report results to the cumulative stats system, too.
@@ -2583,6 +2632,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2601,6 +2654,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2609,6 +2663,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, (void *) vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2633,6 +2694,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2652,12 +2717,20 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3274,7 +3347,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() >= ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3291,7 +3364,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() >= ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3307,16 +3380,22 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index e84d6881403..ee759cdc5c3 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1425,3 +1425,44 @@ WHERE
db.datname = current_database() AND
rel.oid = stats.relid AND
ns.oid = rel.relnamespace;
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_deleted,
+ stats.tuples_deleted,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stat_vacuum_indexes(rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace;
+
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 4e8d8f8dc77..a6f971b5a68 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -851,17 +851,33 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->pages_frozen += src->pages_frozen;
- dst->pages_all_visible += src->pages_all_visible;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->dead_tuples += src->dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
}
/* ------------------------------------------------------------
@@ -1108,7 +1124,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
PG_TRY();
{
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
- pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ if (kind == PGSTAT_KIND_RELATION)
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
}
PG_FINALLY();
{
@@ -1163,6 +1180,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index d40d43cdb4a..5b06b04faad 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -211,7 +211,7 @@ pgstat_drop_relation(Relation rel)
* ---------
*/
void
-pgstat_report_vacuum_error(Oid tableoid)
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -228,6 +228,7 @@ pgstat_report_vacuum_error(Oid tableoid)
tabentry = &shtabentry->stats;
tabentry->vacuum_ext.interrupts++;
+ tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1df271286e6..915c3e59bfa 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2070,17 +2070,19 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
}
#define EXTVACHEAPSTAT_COLUMNS 27
+#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
static void
tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
PgStat_StatTabEntry *tabentry)
{
- Datum values[EXTVACHEAPSTAT_COLUMNS];
- bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ Datum values[EXTVACSTAT_COLUMNS];
+ bool nulls[EXTVACSTAT_COLUMNS];
char buf[256];
int i = 0;
- memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+ memset(nulls, 0, EXTVACSTAT_COLUMNS * sizeof(bool));
values[i++] = ObjectIdGetDatum(relid);
@@ -2093,16 +2095,25 @@ tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
tabentry->vacuum_ext.blks_hit);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
- values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
- values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+ if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_HEAP)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ }
+ else if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.pages_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.tuples_deleted);
+ }
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
@@ -2130,10 +2141,9 @@ tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
* Get the vacuum statistics for the heap tables or indexes.
*/
static void
-pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
{
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
- Oid relid = PG_GETARG_OID(0);
PgStat_StatTabEntry *tabentry;
InitMaterializedSRF(fcinfo, 0);
@@ -2146,35 +2156,37 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Assert(rsinfo->setDesc->natts == ncolumns);
Assert(rsinfo->setResult != NULL);
- /* Load table statistics for specified database. */
- if (OidIsValid(relid))
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
{
- tabentry = pgstat_fetch_stat_tabentry(relid);
- if (tabentry == NULL)
- /* Table don't exists or isn't an heap relation. */
- return;
+ Oid relid = PG_GETARG_OID(0);
- tuplestore_put_for_relation(relid, rsinfo, tabentry);
- }
- else
- {
- SnapshotIterator hashiter;
- PgStat_SnapshotEntry *entry;
-
- /* Iterate the snapshot */
- InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+ /* Load table statistics for specified relation. */
+ if (OidIsValid(relid))
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ if (tabentry == NULL || tabentry->vacuum_ext.type != type)
+ /* Table don't exists or isn't an heap relation. */
+ return;
- while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
+ else
{
- Oid reloid;
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
+
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
- CHECK_FOR_INTERRUPTS();
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
- tabentry = (PgStat_StatTabEntry *) entry->data;
- reloid = entry->key.objoid;
+ tabentry = (PgStat_StatTabEntry *) entry->data;
- if (tabentry != NULL)
- tuplestore_put_for_relation(reloid, rsinfo, tabentry);
+ if (tabentry != NULL && tabentry->vacuum_ext.type == type)
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
}
}
}
@@ -2185,7 +2197,18 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Datum
pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
{
- pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
PG_RETURN_VOID();
}
+
+/*
+ * Get the vacuum statistics for the indexes.
+ */
+Datum
+pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+ }
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2023270f923..604b4f44930 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12263,4 +12263,13 @@
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
prosrc => 'pg_stat_vacuum_tables' },
+{ oid => '8002',
+ descr => 'pg_stat_vacuum_indexes return stats values',
+ proname => 'pg_stat_vacuum_indexes', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_indexes' }
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 4492a0572c6..762b53b88ed 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -167,11 +167,20 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter sync_error_count;
} PgStat_BackendSubEntry;
+
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_HEAP = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -203,14 +212,38 @@ typedef struct ExtVacReport
/* Interruptions on any errors. */
int32 interrupts;
- int64 pages_scanned; /* number of pages we examined */
- int64 pages_removed; /* number of pages removed by vacuum */
- int64 pages_frozen; /* number of pages marked in VM as frozen */
- int64 pages_all_visible; /* number of pages marked in VM as all-visible */
- int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
- int64 index_vacuum_count; /* number of index vacuumings */
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } heap;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -689,7 +722,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 7cdb79c0ec4..93fe15c01f9 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -9,10 +9,9 @@ step s2_print_vacuum_stats_table:
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
-relname |tuples_deleted|dead_tuples|tuples_frozen
---------------------------+--------------+-----------+-------------
-test_vacuum_stat_isolation| 0| 0| 0
-(1 row)
+relname|tuples_deleted|dead_tuples|tuples_frozen
+-------+--------------+-----------+-------------
+(0 rows)
step s1_begin_repeatable_read:
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 7d31ddbece9..bca3e8516b2 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -48,4 +48,4 @@ permutation
s1_commit
s2_checkpoint
s2_vacuum
- s2_print_vacuum_stats_table
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 9ae743eae0c..5d72b970b03 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,10 +32,11 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
-------+-----------------------
+ oid | proname
+------+------------------------
8001 | pg_stat_vacuum_tables
-(1 row)
+ 8002 | pg_stat_vacuum_indexes
+(2 rows)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index cc0b5bde0a1..265eb597d69 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2229,6 +2229,32 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stat_vacuum_indexes(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..a0da8d25f1a
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,158 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+---------+----------+---------------+----------------
+(0 rows)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index 1a7d04b0590..b85a5cab9af 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -37,8 +37,7 @@ FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
---------+--------------+----------------+----------+---------------+---------------
- vestat | 0 | 0 | 455 | 0 | 0
-(1 row)
+(0 rows)
SELECT relpages AS rp
FROM pg_class c
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index f8a4bcccc9d..b9408a43f71 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..9113fd26e6f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,128 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] v6-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (19.9K, 4-v6-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From b33b32ec0fa31a2ed4349adb9e087722cd23484b Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:42:28 +0300
Subject: [PATCH 3/4] Machinery for grabbing an extended vacuum statistics on
databases. It transmits vacuum statistical information about each table and
accumulates it for the database which the table belonged.
---
src/backend/catalog/system_views.sql | 28 +++++++
src/backend/utils/activity/pgstat.c | 2 +
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 16 ++++
src/backend/utils/adt/pgstatfuncs.c | 75 +++++++++++++++++-
src/include/catalog/pg_proc.dat | 11 ++-
src/include/pgstat.h | 3 +-
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/expected/rules.out | 20 ++++-
...ut => vacuum_tables_and_db_statistics.out} | 78 +++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 66 +++++++++++++++-
12 files changed, 300 insertions(+), 9 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (76%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (78%)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ee759cdc5c3..76a2ffff2bb 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1466,3 +1466,31 @@ WHERE
rel.oid = stats.relid AND
ns.oid = rel.relnamespace;
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+
+ stats.interrupts
+FROM
+ pg_database db LEFT JOIN pg_stat_vacuum_database(db.oid) stats
+ON
+ db.oid = stats.dboid;
+
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index a6f971b5a68..b633408777e 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1126,6 +1126,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
if (kind == PGSTAT_KIND_RELATION)
pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ else if (kind == PGSTAT_KIND_DATABASE)
+ pgstat_build_snapshot(PGSTAT_KIND_DATABASE);
}
PG_FINALLY();
{
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 29bc0909748..a060d1a4042 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -430,6 +430,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 5b06b04faad..cc09aba571f 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -217,6 +217,7 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
if (!pgstat_track_counts)
return;
@@ -230,6 +231,10 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
tabentry->vacuum_ext.interrupts++;
tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.interrupts++;
+ dbentry->vacuum_ext.type = m_type;
}
/*
@@ -243,6 +248,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -296,6 +302,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 915c3e59bfa..9a9b6f807bf 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2071,8 +2071,49 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
#define EXTVACHEAPSTAT_COLUMNS 27
#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACDBSTAT_COLUMNS 15
#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
+static void
+tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
+ PgStatShared_Database *dbentry)
+{
+ Datum values[EXTVACDBSTAT_COLUMNS];
+ bool nulls[EXTVACDBSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACDBSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->stats.vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(dbentry->stats.vacuum_ext.interrupts);
+
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+}
+
static void
tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
PgStat_StatTabEntry *tabentry)
@@ -2189,6 +2230,26 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
}
}
}
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ PgStatShared_Database *dbentry;
+ PgStat_EntryRef *entry_ref;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (OidIsValid(dbid))
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dbid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ if (dbentry == NULL)
+ /* Table doesn't exist or isn't a heap relation */
+ return;
+
+ tuplestore_put_for_database(dbid, rsinfo, dbentry);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
}
/*
@@ -2211,4 +2272,16 @@ pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
PG_RETURN_VOID();
- }
\ No newline at end of file
+ }
+
+
+/*
+ * Get the vacuum statistics for the database.
+ */
+Datum
+pg_stat_vacuum_database(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 604b4f44930..d4696d0c055 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12271,5 +12271,14 @@
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
- prosrc => 'pg_stat_vacuum_indexes' }
+ prosrc => 'pg_stat_vacuum_indexes' },
+{ oid => '8003',
+ descr => 'pg_stat_vacuum_database return stats values',
+ proname => 'pg_stat_vacuum_database', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 762b53b88ed..110e9472f3c 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -173,7 +173,8 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_HEAP = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 5d72b970b03..7026de157e4 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,11 +32,12 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
-------+------------------------
+ oid | proname
+------+-------------------------
8001 | pg_stat_vacuum_tables
8002 | pg_stat_vacuum_indexes
-(2 rows)
+ 8003 | pg_stat_vacuum_database
+(3 rows)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 265eb597d69..5d7f73c25fd 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2229,6 +2229,24 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM (pg_database db
+ LEFT JOIN LATERAL pg_stat_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON ((db.oid = stats.dboid)));
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
@@ -2253,7 +2271,7 @@ pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
FROM pg_database db,
pg_class rel,
pg_namespace ns,
- LATERAL pg_stat_vacuum_indexes(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 76%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index b85a5cab9af..ec0cf97e2da 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,6 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -196,4 +199,79 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
t | t | t | t
(1 row)
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index b9408a43f71..129b1102028 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 78%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 41e387dd304..ed9bb852625 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,6 +7,10 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+
-- conditio sine qua non
SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
@@ -155,4 +159,64 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+DROP TABLE vestat CASCADE;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
--
2.34.1
[text/x-patch] v6-0004-Add-documentation-about-the-system-views-that-are-us.patch (24.2K, 5-v6-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 0cbb749c24b3c4ee9891c078a4906ee3f24da762 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:47:55 +0300
Subject: [PATCH 4/4] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 747 +++++++++++++++++++++++++++++++++
1 file changed, 747 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 634a4c0fab4..8cbccdc4a4d 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5064,4 +5064,751 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stats-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this index
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-frozen in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_all_visible</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-visible in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this table
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
+
</chapter>
--
2.34.1
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-25 16:06 Alena Rybakina <[email protected]>
parent: jian he <[email protected]>
1 sibling, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2024-08-25 16:06 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
On 22.08.2024 05:47, jian he wrote:
> On Wed, Aug 21, 2024 at 6:37 AM Alena Rybakina
> <[email protected]> wrote:
>> We check it there: "tabentry->vacuum_ext.type != type". Or were you talking about something else?
>>
>> On 19.08.2024 12:32, jian he wrote:
>>
>> in pg_stats_vacuum
>> if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
>> {
>> Oid relid = PG_GETARG_OID(1);
>>
>> /* Load table statistics for specified database. */
>> if (OidIsValid(relid))
>> {
>> tabentry = fetch_dbstat_tabentry(dbid, relid);
>> if (tabentry == NULL || tabentry->vacuum_ext.type != type)
>> /* Table don't exists or isn't an heap relation. */
>> PG_RETURN_NULL();
>>
>> tuplestore_put_for_relation(relid, rsinfo, tabentry);
>> }
>> else
>> {
>> }
>>
>>
>> So for functions pg_stat_vacuum_indexes and pg_stat_vacuum_tables,
>> it seems you didn't check "relid" 's relkind,
>> you may need to use get_rel_relkind.
>>
>> --
> hi.
> I mentioned some points at [1],
> Please check the attached patchset to address these issues.
Thank you for your work! I checked the patches and added your suggested
changes to the new version of the patch here [0]. In my opinion, nothing
was missing, but please take a look.
[0]
https://www.postgresql.org/message-id/c4e4e305-7119-4183-b49a-d7092f4efba3%40postgrespro.ru
>
> there are four occurrences of "CurrentDatabaseId", i am still confused
> with usage of CurrentDatabaseId.
It needed to be used because of scanning objects from the other
database, so we change the id of dbid temporary to achieve it.
You should snow that every part of this code was deleted.Now we can
check information about tables and indexes from the current database.
> also please don't top-post, otherwise the archive, like [2] is not
> easier to read for future readers.
> generally you quote first, then reply.
>
> [1]https://postgr.es/m/CACJufxHb_YGCp=pVH6DZcpk9yML+SueffPeaRbX2LzXZVahd_w@mail.gmail.com
> [2]https://postgr.es/m/[email protected]
Ok, no problem.
--
Regards,
Alena Rybakina
Postgres Professional:http://www.postgrespro.com
The Russian Postgres Company
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-25 16:12 Alena Rybakina <[email protected]>
parent: Kirill Reshke <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2024-08-25 16:12 UTC (permalink / raw)
To: Kirill Reshke <[email protected]>; +Cc: jian he <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
On 22.08.2024 07:29, Kirill Reshke wrote:
> On Thu, 22 Aug 2024 at 07:48, jian he<[email protected]> wrote:
>> On Wed, Aug 21, 2024 at 6:37 AM Alena Rybakina
>> <[email protected]> wrote:
>>> We check it there: "tabentry->vacuum_ext.type != type". Or were you talking about something else?
>>>
>>> On 19.08.2024 12:32, jian he wrote:
>>>
>>> in pg_stats_vacuum
>>> if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
>>> {
>>> Oid relid = PG_GETARG_OID(1);
>>>
>>> /* Load table statistics for specified database. */
>>> if (OidIsValid(relid))
>>> {
>>> tabentry = fetch_dbstat_tabentry(dbid, relid);
>>> if (tabentry == NULL || tabentry->vacuum_ext.type != type)
>>> /* Table don't exists or isn't an heap relation. */
>>> PG_RETURN_NULL();
>>>
>>> tuplestore_put_for_relation(relid, rsinfo, tabentry);
>>> }
>>> else
>>> {
>>> }
>>>
>>>
>>> So for functions pg_stat_vacuum_indexes and pg_stat_vacuum_tables,
>>> it seems you didn't check "relid" 's relkind,
>>> you may need to use get_rel_relkind.
>>>
>>> --
>> hi.
>> I mentioned some points at [1],
>> Please check the attached patchset to address these issues.
>>
>> there are four occurrences of "CurrentDatabaseId", i am still confused
>> with usage of CurrentDatabaseId.
>>
>> also please don't top-post, otherwise the archive, like [2] is not
>> easier to read for future readers.
>> generally you quote first, then reply.
>>
>> [1]https://postgr.es/m/CACJufxHb_YGCp=pVH6DZcpk9yML+SueffPeaRbX2LzXZVahd_w@mail.gmail.com
>> [2]https://postgr.es/m/[email protected]
> Hi, your points are valid.
> Regarding 0003, I also wanted to object database naming in a
> regression test during my review but for some reason didn't.Now, as
> soon as we already need to change it, I suggest we also change
> regression_statistic_vacuum_db1 to something less generic. Maybe
> regression_statistic_vacuum_db_unaffected.
>
Hi! I fixed it in the new version of the patch [0]. Feel free to review it!
To be honest, I still doubt that the current database names
(regression_statistic_vacuum_db and regression_statistic_vacuum_db1) can
be easily generated, but if you insist on renaming, I will do it.
[0]
https://www.postgresql.org/message-id/c4e4e305-7119-4183-b49a-d7092f4efba3%40postgrespro.ru
--
Regards,
Alena Rybakina
Postgres Professional:http://www.postgrespro.com
The Russian Postgres Company
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-08-26 11:55 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
2 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2024-08-26 11:55 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; jian he <[email protected]>; Kirill Reshke <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
Just in case, I have attached a diff file to show the changes for the
latest version attached here [0] to make the review process easier.
[0]
https://www.postgresql.org/message-id/c4e4e305-7119-4183-b49a-d7092f4efba3%40postgrespro.ru
--
Regards,
Alena Rybakina
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 42d3ad21486..8cbccdc4a4d 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5360,7 +5360,7 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
Number of times blocks of this index were already found
in the buffer cache by vacuum operations, so that a read was not necessary
(this only includes hits in the
- &project; buffer cache, not the operating system's file system cache)
+ project; buffer cache, not the operating system's file system cache)
</para></entry>
</row>
@@ -5601,7 +5601,7 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
Number of times blocks of this table were already found
in the buffer cache by vacuum operations, so that a read was not necessary
(this only includes hits in the
- &project; buffer cache, not the operating system's file system cache)
+ project; buffer cache, not the operating system's file system cache)
</para></entry>
</row>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ca3ad09727e..76a2ffff2bb 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1420,7 +1420,7 @@ FROM
pg_database db,
pg_class rel,
pg_namespace ns,
- pg_stat_vacuum_tables(db.oid, rel.oid) stats
+ pg_stat_vacuum_tables(rel.oid) stats
WHERE
db.datname = current_database() AND
rel.oid = stats.relid AND
@@ -1460,7 +1460,7 @@ FROM
pg_database db,
pg_class rel,
pg_namespace ns,
- pg_stat_vacuum_indexes(db.oid, rel.oid) stats
+ pg_stat_vacuum_indexes(rel.oid) stats
WHERE
db.datname = current_database() AND
rel.oid = stats.relid AND
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 3c50bea379c..b633408777e 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -146,6 +146,34 @@
#define PGSTAT_FILE_ENTRY_HASH 'S' /* stats entry identified by
* PgStat_HashKey */
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
/* ----------
* Local function forward declarations
@@ -232,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 0c490ba5f1a..9a9b6f807bf 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -33,6 +33,41 @@
#include "utils/timestamp.h"
#include "utils/pgstat_internal.h"
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
+
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
#define HAS_PGSTAT_PERMISSIONS(role) (has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS) || has_privs_of_role(GetUserId(), role))
@@ -2039,47 +2074,9 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
#define EXTVACDBSTAT_COLUMNS 15
#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
-static Oid CurrentDatabaseId = InvalidOid;
-
-
-/*
- * Fetch stat collector data for specific database and table, which loading from disc.
- * It is maybe expensive, but i guess we won't use that machinery often.
- * The kind of bufferization is based on CurrentDatabaseId value.
- */
-static PgStat_StatTabEntry *
-fetch_dbstat_tabentry(Oid dbid, Oid relid)
-{
- Oid storedMyDatabaseId = MyDatabaseId;
- PgStat_StatTabEntry *tabentry = NULL;
-
- if (OidIsValid(CurrentDatabaseId) && CurrentDatabaseId == dbid)
- /* Quick path when we read data from the same database */
- return pgstat_fetch_stat_tabentry(relid);
-
- pgstat_clear_snapshot();
-
- /* Tricky turn here: enforce pgstat to think that our database has dbid */
-
- MyDatabaseId = dbid;
-
- PG_TRY();
- {
- tabentry = pgstat_fetch_stat_tabentry(relid);
- MyDatabaseId = storedMyDatabaseId;
- }
- PG_CATCH();
- {
- MyDatabaseId = storedMyDatabaseId;
- }
- PG_END_TRY();
-
- return tabentry;
-}
-
static void
-tuplestore_put_for_database(Oid dbid, Tuplestorestate *tupstore,
- TupleDesc tupdesc, PgStatShared_Database *dbentry, int ncolumns)
+tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
+ PgStatShared_Database *dbentry)
{
Datum values[EXTVACDBSTAT_COLUMNS];
bool nulls[EXTVACDBSTAT_COLUMNS];
@@ -2113,15 +2110,13 @@ tuplestore_put_for_database(Oid dbid, Tuplestorestate *tupstore,
values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.total_time);
values[i++] = Int32GetDatum(dbentry->stats.vacuum_ext.interrupts);
-
- Assert(i == ncolumns);
-
- tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
}
static void
-tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
- TupleDesc tupdesc, PgStat_StatTabEntry *tabentry, int ncolumns)
+tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
+ PgStat_StatTabEntry *tabentry)
{
Datum values[EXTVACSTAT_COLUMNS];
bool nulls[EXTVACSTAT_COLUMNS];
@@ -2179,23 +2174,17 @@ tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
values[i++] = Float8GetDatum(tabentry->vacuum_ext.total_time);
values[i++] = Int32GetDatum(tabentry->vacuum_ext.interrupts);
- Assert(i == ncolumns);
-
- tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
}
/*
* Get the vacuum statistics for the heap tables or indexes.
*/
-static Datum
+static void
pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
{
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
- MemoryContext per_query_ctx;
- MemoryContext oldcontext;
- Tuplestorestate *tupstore;
- TupleDesc tupdesc;
- Oid dbid = PG_GETARG_OID(0);
PgStat_StatTabEntry *tabentry;
InitMaterializedSRF(fcinfo, 0);
@@ -2205,61 +2194,39 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("set-valued function called in context that cannot accept a set")));
- /* Switch to long-lived context to create the returned data structures */
- per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
- oldcontext = MemoryContextSwitchTo(per_query_ctx);
-
- /* Build a tuple descriptor for our result type */
- if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
-
- Assert(tupdesc->natts == ncolumns);
-
- tupstore = tuplestore_begin_heap(true, false, work_mem);
- Assert (tupstore != NULL);
- rsinfo->setResult = tupstore;
- rsinfo->setDesc = tupdesc;
-
- MemoryContextSwitchTo(oldcontext);
+ Assert(rsinfo->setDesc->natts == ncolumns);
+ Assert(rsinfo->setResult != NULL);
if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
{
- Oid relid = PG_GETARG_OID(1);
+ Oid relid = PG_GETARG_OID(0);
- /* Load table statistics for specified database. */
+ /* Load table statistics for specified relation. */
if (OidIsValid(relid))
{
- tabentry = fetch_dbstat_tabentry(dbid, relid);
+ tabentry = pgstat_fetch_stat_tabentry(relid);
if (tabentry == NULL || tabentry->vacuum_ext.type != type)
/* Table don't exists or isn't an heap relation. */
- PG_RETURN_NULL();
+ return;
- tuplestore_put_for_relation(relid, tupstore, tupdesc, tabentry, ncolumns);
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
}
else
{
SnapshotIterator hashiter;
PgStat_SnapshotEntry *entry;
- Oid storedMyDatabaseId = MyDatabaseId;
-
- pgstat_update_snapshot(PGSTAT_KIND_RELATION);
- MyDatabaseId = storedMyDatabaseId;
-
/* Iterate the snapshot */
InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
{
- Oid reloid;
-
CHECK_FOR_INTERRUPTS();
tabentry = (PgStat_StatTabEntry *) entry->data;
- reloid = entry->key.objoid;
if (tabentry != NULL && tabentry->vacuum_ext.type == type)
- tuplestore_put_for_relation(reloid, tupstore, tupdesc, tabentry, ncolumns);
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
}
}
}
@@ -2267,10 +2234,7 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
{
PgStatShared_Database *dbentry;
PgStat_EntryRef *entry_ref;
- Oid storedMyDatabaseId = MyDatabaseId;
-
- pgstat_update_snapshot(PGSTAT_KIND_DATABASE);
- MyDatabaseId = storedMyDatabaseId;
+ Oid dbid = PG_GETARG_OID(0);
if (OidIsValid(dbid))
{
@@ -2280,15 +2244,12 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
if (dbentry == NULL)
/* Table doesn't exist or isn't a heap relation */
- PG_RETURN_NULL();
+ return;
- tuplestore_put_for_database(dbid, tupstore, tupdesc, dbentry, ncolumns);
+ tuplestore_put_for_database(dbid, rsinfo, dbentry);
pgstat_unlock_entry(entry_ref);
}
- else
- PG_RETURN_NULL();
}
- PG_RETURN_NULL();
}
/*
@@ -2297,9 +2258,9 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
Datum
pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
{
- return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
- PG_RETURN_NULL();
+ PG_RETURN_VOID();
}
/*
@@ -2308,10 +2269,11 @@ pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
Datum
pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
{
- return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+ }
- PG_RETURN_NULL();
-}
/*
* Get the vacuum statistics for the database.
@@ -2319,7 +2281,7 @@ pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
Datum
pg_stat_vacuum_database(PG_FUNCTION_ARGS)
{
- return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
- PG_RETURN_NULL();
-}
+ PG_RETURN_VOID();
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b2e881aa89d..d4696d0c055 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12258,20 +12258,20 @@
descr => 'pg_stat_vacuum_tables return stats values',
proname => 'pg_stat_vacuum_tables', provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
- proargtypes => 'oid oid',
- proallargtypes => '{oid,oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
- proargmodes => '{i,i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{dboid,reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
prosrc => 'pg_stat_vacuum_tables' },
{ oid => '8002',
descr => 'pg_stat_vacuum_indexes return stats values',
proname => 'pg_stat_vacuum_indexes', provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
- proargtypes => 'oid oid',
- proallargtypes => '{oid,oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
- proargmodes => '{i,i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{dboid,reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
- prosrc => 'pg_stat_vacuum_indexes' }
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_indexes' },
{ oid => '8003',
descr => 'pg_stat_vacuum_database return stats values',
proname => 'pg_stat_vacuum_database', provolatile => 's', prorettype => 'record',proisstrict => 'f',
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 715ae1b6fd4..24ab3ceb717 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -874,38 +874,4 @@ pgstat_get_custom_snapshot_data(PgStat_Kind kind)
return pgStatLocal.snapshot.custom_data[idx];
}
-/* hash table for statistics snapshots entry */
-typedef struct PgStat_SnapshotEntry
-{
- PgStat_HashKey key;
- char status; /* for simplehash use */
- void *data; /* the stats data itself */
-} PgStat_SnapshotEntry;
-
-/* ----------
- * Backend-local Hash Table Definitions
- * ----------
- */
-
-/* for stats snapshot entries */
-#define SH_PREFIX pgstat_snapshot
-#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
-#define SH_KEY_TYPE PgStat_HashKey
-#define SH_KEY key
-#define SH_HASH_KEY(tb, key) \
- pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
-#define SH_EQUAL(tb, a, b) \
- pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
-#define SH_SCOPE static inline
-#define SH_DEFINE
-#define SH_DECLARE
-#include "lib/simplehash.h"
-
-typedef pgstat_snapshot_iterator SnapshotIterator;
-
-#define InitSnapshotIterator(htable, iter) \
- pgstat_snapshot_start_iterate(htable, iter);
-#define ScanStatSnapshot(htable, iter) \
- pgstat_snapshot_iterate(htable, iter)
-
#endif /* PGSTAT_INTERNAL_H */
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index c4388dd0da1..5d7f73c25fd 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2271,7 +2271,7 @@ pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
FROM pg_database db,
pg_class rel,
pg_namespace ns,
- LATERAL pg_stat_vacuum_indexes(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
@@ -2305,7 +2305,7 @@ pg_stat_vacuum_tables| SELECT rel.oid AS relid,
FROM pg_database db,
pg_class rel,
pg_namespace ns,
- LATERAL pg_stat_vacuum_tables(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_wal| SELECT wal_records,
wal_fpi,
diff --git a/src/test/regress/expected/vacuum_tables_and_db_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
index f0537aac430..ec0cf97e2da 100644
--- a/src/test/regress/expected/vacuum_tables_and_db_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,9 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
-CREATE DATABASE statistic_vacuum_database;
-CREATE DATABASE statistic_vacuum_database1;
-\c statistic_vacuum_database;
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -212,9 +212,9 @@ SELECT dbname,
FROM
pg_stat_vacuum_database
WHERE dbname = current_database();
- dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
----------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
- statistic_vacuum_database | t | t | t | t | t | t | t | t
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
(1 row)
DROP TABLE vestat CASCADE;
@@ -230,7 +230,7 @@ INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
UPDATE vestat SET x = 10001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-\c statistic_vacuum_database1;
+\c regression_statistic_vacuum_db1;
-- Now check vacuum statistics for postgres database from another database
SELECT dbname,
db_blks_hit > 0 AS db_blks_hit,
@@ -243,20 +243,20 @@ SELECT dbname,
total_time > 0 AS total_time
FROM
pg_stat_vacuum_database
-WHERE dbname = 'statistic_vacuum_database';
- dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
----------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
- statistic_vacuum_database | t | t | t | t | t | t | t | t
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
(1 row)
-\c statistic_vacuum_database
+\c regression_statistic_vacuum_db
RESET vacuum_freeze_min_age;
RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
-\c statistic_vacuum_database1;
+\c regression_statistic_vacuum_db1;
SELECT count(*)
FROM pg_database d
-CROSS JOIN pg_stat_vacuum_tables(d.oid, 0)
+CROSS JOIN pg_stat_vacuum_tables(0)
WHERE oid = 0; -- must be 0
count
-------
@@ -273,5 +273,5 @@ WHERE oid = 0; -- must be 0
(1 row)
\c postgres
-DROP DATABASE statistic_vacuum_database1;
-DROP DATABASE statistic_vacuum_database;
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 43cc8068b0f..ed9bb852625 100644
--- a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,9 +7,9 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
-CREATE DATABASE statistic_vacuum_database;
-CREATE DATABASE statistic_vacuum_database1;
-\c statistic_vacuum_database;
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
@@ -184,7 +184,7 @@ ANALYZE vestat;
UPDATE vestat SET x = 10001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-\c statistic_vacuum_database1;
+\c regression_statistic_vacuum_db1;
-- Now check vacuum statistics for postgres database from another database
SELECT dbname,
@@ -198,18 +198,18 @@ SELECT dbname,
total_time > 0 AS total_time
FROM
pg_stat_vacuum_database
-WHERE dbname = 'statistic_vacuum_database';
+WHERE dbname = 'regression_statistic_vacuum_db';
-\c statistic_vacuum_database
+\c regression_statistic_vacuum_db
RESET vacuum_freeze_min_age;
RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
-\c statistic_vacuum_database1;
+\c regression_statistic_vacuum_db1;
SELECT count(*)
FROM pg_database d
-CROSS JOIN pg_stat_vacuum_tables(d.oid, 0)
+CROSS JOIN pg_stat_vacuum_tables(0)
WHERE oid = 0; -- must be 0
SELECT count(*)
@@ -218,5 +218,5 @@ CROSS JOIN pg_stat_vacuum_database(0)
WHERE oid = 0; -- must be 0
\c postgres
-DROP DATABASE statistic_vacuum_database1;
-DROP DATABASE statistic_vacuum_database;
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
Attachments:
[text/plain] diff_vacuum.diff.no-cfbot (24.4K, 2-diff_vacuum.diff.no-cfbot)
download | inline diff:
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 42d3ad21486..8cbccdc4a4d 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5360,7 +5360,7 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
Number of times blocks of this index were already found
in the buffer cache by vacuum operations, so that a read was not necessary
(this only includes hits in the
- &project; buffer cache, not the operating system's file system cache)
+ project; buffer cache, not the operating system's file system cache)
</para></entry>
</row>
@@ -5601,7 +5601,7 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
Number of times blocks of this table were already found
in the buffer cache by vacuum operations, so that a read was not necessary
(this only includes hits in the
- &project; buffer cache, not the operating system's file system cache)
+ project; buffer cache, not the operating system's file system cache)
</para></entry>
</row>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ca3ad09727e..76a2ffff2bb 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1420,7 +1420,7 @@ FROM
pg_database db,
pg_class rel,
pg_namespace ns,
- pg_stat_vacuum_tables(db.oid, rel.oid) stats
+ pg_stat_vacuum_tables(rel.oid) stats
WHERE
db.datname = current_database() AND
rel.oid = stats.relid AND
@@ -1460,7 +1460,7 @@ FROM
pg_database db,
pg_class rel,
pg_namespace ns,
- pg_stat_vacuum_indexes(db.oid, rel.oid) stats
+ pg_stat_vacuum_indexes(rel.oid) stats
WHERE
db.datname = current_database() AND
rel.oid = stats.relid AND
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 3c50bea379c..b633408777e 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -146,6 +146,34 @@
#define PGSTAT_FILE_ENTRY_HASH 'S' /* stats entry identified by
* PgStat_HashKey */
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
/* ----------
* Local function forward declarations
@@ -232,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 0c490ba5f1a..9a9b6f807bf 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -33,6 +33,41 @@
#include "utils/timestamp.h"
#include "utils/pgstat_internal.h"
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
+
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
#define HAS_PGSTAT_PERMISSIONS(role) (has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS) || has_privs_of_role(GetUserId(), role))
@@ -2039,47 +2074,9 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
#define EXTVACDBSTAT_COLUMNS 15
#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
-static Oid CurrentDatabaseId = InvalidOid;
-
-
-/*
- * Fetch stat collector data for specific database and table, which loading from disc.
- * It is maybe expensive, but i guess we won't use that machinery often.
- * The kind of bufferization is based on CurrentDatabaseId value.
- */
-static PgStat_StatTabEntry *
-fetch_dbstat_tabentry(Oid dbid, Oid relid)
-{
- Oid storedMyDatabaseId = MyDatabaseId;
- PgStat_StatTabEntry *tabentry = NULL;
-
- if (OidIsValid(CurrentDatabaseId) && CurrentDatabaseId == dbid)
- /* Quick path when we read data from the same database */
- return pgstat_fetch_stat_tabentry(relid);
-
- pgstat_clear_snapshot();
-
- /* Tricky turn here: enforce pgstat to think that our database has dbid */
-
- MyDatabaseId = dbid;
-
- PG_TRY();
- {
- tabentry = pgstat_fetch_stat_tabentry(relid);
- MyDatabaseId = storedMyDatabaseId;
- }
- PG_CATCH();
- {
- MyDatabaseId = storedMyDatabaseId;
- }
- PG_END_TRY();
-
- return tabentry;
-}
-
static void
-tuplestore_put_for_database(Oid dbid, Tuplestorestate *tupstore,
- TupleDesc tupdesc, PgStatShared_Database *dbentry, int ncolumns)
+tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
+ PgStatShared_Database *dbentry)
{
Datum values[EXTVACDBSTAT_COLUMNS];
bool nulls[EXTVACDBSTAT_COLUMNS];
@@ -2113,15 +2110,13 @@ tuplestore_put_for_database(Oid dbid, Tuplestorestate *tupstore,
values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.total_time);
values[i++] = Int32GetDatum(dbentry->stats.vacuum_ext.interrupts);
-
- Assert(i == ncolumns);
-
- tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
}
static void
-tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
- TupleDesc tupdesc, PgStat_StatTabEntry *tabentry, int ncolumns)
+tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
+ PgStat_StatTabEntry *tabentry)
{
Datum values[EXTVACSTAT_COLUMNS];
bool nulls[EXTVACSTAT_COLUMNS];
@@ -2179,23 +2174,17 @@ tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
values[i++] = Float8GetDatum(tabentry->vacuum_ext.total_time);
values[i++] = Int32GetDatum(tabentry->vacuum_ext.interrupts);
- Assert(i == ncolumns);
-
- tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
}
/*
* Get the vacuum statistics for the heap tables or indexes.
*/
-static Datum
+static void
pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
{
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
- MemoryContext per_query_ctx;
- MemoryContext oldcontext;
- Tuplestorestate *tupstore;
- TupleDesc tupdesc;
- Oid dbid = PG_GETARG_OID(0);
PgStat_StatTabEntry *tabentry;
InitMaterializedSRF(fcinfo, 0);
@@ -2205,61 +2194,39 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("set-valued function called in context that cannot accept a set")));
- /* Switch to long-lived context to create the returned data structures */
- per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
- oldcontext = MemoryContextSwitchTo(per_query_ctx);
-
- /* Build a tuple descriptor for our result type */
- if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
-
- Assert(tupdesc->natts == ncolumns);
-
- tupstore = tuplestore_begin_heap(true, false, work_mem);
- Assert (tupstore != NULL);
- rsinfo->setResult = tupstore;
- rsinfo->setDesc = tupdesc;
-
- MemoryContextSwitchTo(oldcontext);
+ Assert(rsinfo->setDesc->natts == ncolumns);
+ Assert(rsinfo->setResult != NULL);
if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
{
- Oid relid = PG_GETARG_OID(1);
+ Oid relid = PG_GETARG_OID(0);
- /* Load table statistics for specified database. */
+ /* Load table statistics for specified relation. */
if (OidIsValid(relid))
{
- tabentry = fetch_dbstat_tabentry(dbid, relid);
+ tabentry = pgstat_fetch_stat_tabentry(relid);
if (tabentry == NULL || tabentry->vacuum_ext.type != type)
/* Table don't exists or isn't an heap relation. */
- PG_RETURN_NULL();
+ return;
- tuplestore_put_for_relation(relid, tupstore, tupdesc, tabentry, ncolumns);
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
}
else
{
SnapshotIterator hashiter;
PgStat_SnapshotEntry *entry;
- Oid storedMyDatabaseId = MyDatabaseId;
-
- pgstat_update_snapshot(PGSTAT_KIND_RELATION);
- MyDatabaseId = storedMyDatabaseId;
-
/* Iterate the snapshot */
InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
{
- Oid reloid;
-
CHECK_FOR_INTERRUPTS();
tabentry = (PgStat_StatTabEntry *) entry->data;
- reloid = entry->key.objoid;
if (tabentry != NULL && tabentry->vacuum_ext.type == type)
- tuplestore_put_for_relation(reloid, tupstore, tupdesc, tabentry, ncolumns);
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
}
}
}
@@ -2267,10 +2234,7 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
{
PgStatShared_Database *dbentry;
PgStat_EntryRef *entry_ref;
- Oid storedMyDatabaseId = MyDatabaseId;
-
- pgstat_update_snapshot(PGSTAT_KIND_DATABASE);
- MyDatabaseId = storedMyDatabaseId;
+ Oid dbid = PG_GETARG_OID(0);
if (OidIsValid(dbid))
{
@@ -2280,15 +2244,12 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
if (dbentry == NULL)
/* Table doesn't exist or isn't a heap relation */
- PG_RETURN_NULL();
+ return;
- tuplestore_put_for_database(dbid, tupstore, tupdesc, dbentry, ncolumns);
+ tuplestore_put_for_database(dbid, rsinfo, dbentry);
pgstat_unlock_entry(entry_ref);
}
- else
- PG_RETURN_NULL();
}
- PG_RETURN_NULL();
}
/*
@@ -2297,9 +2258,9 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
Datum
pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
{
- return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
- PG_RETURN_NULL();
+ PG_RETURN_VOID();
}
/*
@@ -2308,10 +2269,11 @@ pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
Datum
pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
{
- return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+ }
- PG_RETURN_NULL();
-}
/*
* Get the vacuum statistics for the database.
@@ -2319,7 +2281,7 @@ pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
Datum
pg_stat_vacuum_database(PG_FUNCTION_ARGS)
{
- return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
- PG_RETURN_NULL();
-}
+ PG_RETURN_VOID();
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b2e881aa89d..d4696d0c055 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12258,20 +12258,20 @@
descr => 'pg_stat_vacuum_tables return stats values',
proname => 'pg_stat_vacuum_tables', provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
- proargtypes => 'oid oid',
- proallargtypes => '{oid,oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
- proargmodes => '{i,i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{dboid,reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
prosrc => 'pg_stat_vacuum_tables' },
{ oid => '8002',
descr => 'pg_stat_vacuum_indexes return stats values',
proname => 'pg_stat_vacuum_indexes', provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
- proargtypes => 'oid oid',
- proallargtypes => '{oid,oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
- proargmodes => '{i,i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{dboid,reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
- prosrc => 'pg_stat_vacuum_indexes' }
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_indexes' },
{ oid => '8003',
descr => 'pg_stat_vacuum_database return stats values',
proname => 'pg_stat_vacuum_database', provolatile => 's', prorettype => 'record',proisstrict => 'f',
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 715ae1b6fd4..24ab3ceb717 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -874,38 +874,4 @@ pgstat_get_custom_snapshot_data(PgStat_Kind kind)
return pgStatLocal.snapshot.custom_data[idx];
}
-/* hash table for statistics snapshots entry */
-typedef struct PgStat_SnapshotEntry
-{
- PgStat_HashKey key;
- char status; /* for simplehash use */
- void *data; /* the stats data itself */
-} PgStat_SnapshotEntry;
-
-/* ----------
- * Backend-local Hash Table Definitions
- * ----------
- */
-
-/* for stats snapshot entries */
-#define SH_PREFIX pgstat_snapshot
-#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
-#define SH_KEY_TYPE PgStat_HashKey
-#define SH_KEY key
-#define SH_HASH_KEY(tb, key) \
- pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
-#define SH_EQUAL(tb, a, b) \
- pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
-#define SH_SCOPE static inline
-#define SH_DEFINE
-#define SH_DECLARE
-#include "lib/simplehash.h"
-
-typedef pgstat_snapshot_iterator SnapshotIterator;
-
-#define InitSnapshotIterator(htable, iter) \
- pgstat_snapshot_start_iterate(htable, iter);
-#define ScanStatSnapshot(htable, iter) \
- pgstat_snapshot_iterate(htable, iter)
-
#endif /* PGSTAT_INTERNAL_H */
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index c4388dd0da1..5d7f73c25fd 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2271,7 +2271,7 @@ pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
FROM pg_database db,
pg_class rel,
pg_namespace ns,
- LATERAL pg_stat_vacuum_indexes(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
@@ -2305,7 +2305,7 @@ pg_stat_vacuum_tables| SELECT rel.oid AS relid,
FROM pg_database db,
pg_class rel,
pg_namespace ns,
- LATERAL pg_stat_vacuum_tables(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_wal| SELECT wal_records,
wal_fpi,
diff --git a/src/test/regress/expected/vacuum_tables_and_db_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
index f0537aac430..ec0cf97e2da 100644
--- a/src/test/regress/expected/vacuum_tables_and_db_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,9 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
-CREATE DATABASE statistic_vacuum_database;
-CREATE DATABASE statistic_vacuum_database1;
-\c statistic_vacuum_database;
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -212,9 +212,9 @@ SELECT dbname,
FROM
pg_stat_vacuum_database
WHERE dbname = current_database();
- dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
----------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
- statistic_vacuum_database | t | t | t | t | t | t | t | t
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
(1 row)
DROP TABLE vestat CASCADE;
@@ -230,7 +230,7 @@ INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
UPDATE vestat SET x = 10001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-\c statistic_vacuum_database1;
+\c regression_statistic_vacuum_db1;
-- Now check vacuum statistics for postgres database from another database
SELECT dbname,
db_blks_hit > 0 AS db_blks_hit,
@@ -243,20 +243,20 @@ SELECT dbname,
total_time > 0 AS total_time
FROM
pg_stat_vacuum_database
-WHERE dbname = 'statistic_vacuum_database';
- dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
----------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
- statistic_vacuum_database | t | t | t | t | t | t | t | t
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
(1 row)
-\c statistic_vacuum_database
+\c regression_statistic_vacuum_db
RESET vacuum_freeze_min_age;
RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
-\c statistic_vacuum_database1;
+\c regression_statistic_vacuum_db1;
SELECT count(*)
FROM pg_database d
-CROSS JOIN pg_stat_vacuum_tables(d.oid, 0)
+CROSS JOIN pg_stat_vacuum_tables(0)
WHERE oid = 0; -- must be 0
count
-------
@@ -273,5 +273,5 @@ WHERE oid = 0; -- must be 0
(1 row)
\c postgres
-DROP DATABASE statistic_vacuum_database1;
-DROP DATABASE statistic_vacuum_database;
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 43cc8068b0f..ed9bb852625 100644
--- a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,9 +7,9 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
-CREATE DATABASE statistic_vacuum_database;
-CREATE DATABASE statistic_vacuum_database1;
-\c statistic_vacuum_database;
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
@@ -184,7 +184,7 @@ ANALYZE vestat;
UPDATE vestat SET x = 10001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-\c statistic_vacuum_database1;
+\c regression_statistic_vacuum_db1;
-- Now check vacuum statistics for postgres database from another database
SELECT dbname,
@@ -198,18 +198,18 @@ SELECT dbname,
total_time > 0 AS total_time
FROM
pg_stat_vacuum_database
-WHERE dbname = 'statistic_vacuum_database';
+WHERE dbname = 'regression_statistic_vacuum_db';
-\c statistic_vacuum_database
+\c regression_statistic_vacuum_db
RESET vacuum_freeze_min_age;
RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
-\c statistic_vacuum_database1;
+\c regression_statistic_vacuum_db1;
SELECT count(*)
FROM pg_database d
-CROSS JOIN pg_stat_vacuum_tables(d.oid, 0)
+CROSS JOIN pg_stat_vacuum_tables(0)
WHERE oid = 0; -- must be 0
SELECT count(*)
@@ -218,5 +218,5 @@ CROSS JOIN pg_stat_vacuum_database(0)
WHERE oid = 0; -- must be 0
\c postgres
-DROP DATABASE statistic_vacuum_database1;
-DROP DATABASE statistic_vacuum_database;
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-09-04 17:23 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
2 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2024-09-04 17:23 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; jian he <[email protected]>; Ilia Evdokimov <[email protected]>; +Cc: Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
Hi, all!
I noticed that the pgstat_accumulate_extvac_stats function may be
declared as static in the pgstat_relation.c file rather than in the
pgstat.h file.
I fixed part of the code with interrupt counters. I believe that it is
not worth taking into account the number of interrupts if its level is
greater than ERROR, for example PANIC. Our server will no longer be
available to us and statistics data will not help us.
I have attached the new version of the code and the diff files
(minor-vacuum.no-cbot).
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 4e2ae78d255..9c53d0b4c57 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -3346,7 +3346,7 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
- if(geterrelevel() >= ERROR)
+ if(geterrelevel() == ERROR)
pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -3363,7 +3363,7 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
- if(geterrelevel() >= ERROR)
+ if(geterrelevel() == ERROR)
pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -3380,21 +3380,21 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
- if(geterrelevel() >= ERROR)
+ if(geterrelevel() == ERROR)
pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
- if(geterrelevel() >= ERROR)
+ if(geterrelevel() == ERROR)
pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
- if(geterrelevel() >= ERROR)
+ if(geterrelevel() == ERROR)
pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index b633408777e..583c3ff0f03 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -829,57 +829,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info)
-{
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->system_time += src->system_time;
- dst->user_time += src->user_time;
- dst->total_time += src->total_time;
- dst->interrupts += src->interrupts;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_HEAP)
- {
- dst->heap.pages_scanned += src->heap.pages_scanned;
- dst->heap.pages_removed += src->heap.pages_removed;
- dst->heap.pages_frozen += src->heap.pages_frozen;
- dst->heap.pages_all_visible += src->heap.pages_all_visible;
- dst->heap.tuples_deleted += src->heap.tuples_deleted;
- dst->heap.tuples_frozen += src->heap.tuples_frozen;
- dst->heap.dead_tuples += src->heap.dead_tuples;
- dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->index.tuples_deleted += src->index.tuples_deleted;
- }
- }
-}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index cc09aba571f..e05de63b2f0 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -48,6 +48,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -1034,3 +1036,66 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->system_time += src->system_time;
+ dst->user_time += src->user_time;
+ dst->total_time += src->total_time;
+ dst->interrupts += src->interrupts;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->dead_tuples += src->dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
+}
\ No newline at end of file
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index c56c54de3b4..eacbee579b3 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -646,7 +646,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -721,9 +721,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-extern void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
* Functions in pgstat_replslot.c
Attachments:
[text/x-patch] v7-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (64.1K, 2-v7-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 1247c8da4964b2426d90195e824e3b8207b8bff3 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Wed, 4 Sep 2024 18:52:40 +0300
Subject: [PATCH 1/3] Machinery for grabbing an extended vacuum statistics on
heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 159 +++++++++++++-
src/backend/access/heap/visibilitymap.c | 13 ++
src/backend/catalog/system_views.sql | 54 +++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 32 ++-
src/backend/utils/activity/pgstat_relation.c | 72 ++++++-
src/backend/utils/adt/pgstatfuncs.c | 157 ++++++++++++++
src/backend/utils/error/elog.c | 13 ++
src/include/catalog/pg_proc.dat | 10 +-
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 81 ++++++-
src/include/utils/elog.h | 1 +
src/include/utils/pgstat_internal.h | 2 +-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 51 +++++
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/expected/rules.out | 34 +++
.../expected/vacuum_tables_statistics.out | 200 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 158 ++++++++++++++
22 files changed, 1092 insertions(+), 17 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d82aa3d4896..d63303c7fb7 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -167,6 +167,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -194,6 +195,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -226,6 +229,22 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Cut-off values of parameters which changes implicitly during a vacuum
+ * process.
+ * Vacuum can't control their values, so we should store them before and after
+ * the processing.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz time;
+ PGRUsage ru;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -279,6 +298,115 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+ PGRUsage ru0;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ pg_rusage_init(&ru0);
+ starttime = GetCurrentTimestamp();
+
+ counters->ru = ru0;
+ counters->time = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+ PGRUsage ru1;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->time, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ /*
+ * Get difference of a system time and user time values in milliseconds.
+ * Use floating point representation to show tails of time diffs.
+ */
+ pg_rusage_init(&ru1);
+ report->system_time =
+ (ru1.ru.ru_stime.tv_sec - counters->ru.ru.ru_stime.tv_sec) * 1000. +
+ (ru1.ru.ru_stime.tv_usec - counters->ru.ru.ru_stime.tv_usec) * 0.001;
+ report->user_time =
+ (ru1.ru.ru_utime.tv_sec - counters->ru.ru.ru_utime.tv_sec) * 1000. +
+ (ru1.ru.ru_utime.tv_usec - counters->ru.ru.ru_utime.tv_usec) * 0.001;
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -311,6 +439,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -329,7 +459,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -346,6 +476,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -413,6 +544,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -574,6 +707,19 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -588,7 +734,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1380,6 +1527,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2277,11 +2426,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -3122,6 +3273,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3137,6 +3290,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33c..d72cade60a4 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,18 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Initially, it didn't matter what type of flags (all-visible or frozen) we received,
+ * we just performed a reverse concatenation operation. But this information is very important
+ * for vacuum statistics. We need to find out this usingthe bit concatenation operation
+ * with the VISIBILITYMAP_ALL_VISIBLE and VISIBILITYMAP_ALL_FROZEN masks,
+ * and where the desired one matches, we increment the value there.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 7fd5d256a18..31be7f04476 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1377,3 +1377,57 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stat_vacuum_tables(rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace;
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 7d8e9d20454..363924d00db 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -103,6 +103,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2394,6 +2397,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 22c057fe61b..13ab633086a 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1043,6 +1043,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index b2ca3f39b7a..808a5f15c82 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -260,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -830,7 +829,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -896,7 +894,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, Oid objoid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1012,7 +1010,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1062,8 +1060,30 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 8a3f7d434cf..791d777fbc6 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -48,6 +48,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -204,12 +206,40 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.interrupts++;
+ pgstat_unlock_entry(entry_ref);
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -233,6 +263,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -861,6 +893,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -984,3 +1019,38 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->system_time += src->system_time;
+ dst->user_time += src->user_time;
+ dst->total_time += src->total_time;
+ dst->interrupts += src->interrupts;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->dead_tuples += src->dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 97dc09ac0d9..0966c0bf28b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -31,6 +31,42 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+#include "utils/pgstat_internal.h"
+
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -2051,3 +2087,124 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objoid));
}
+
+#define EXTVACHEAPSTAT_COLUMNS 27
+
+static void
+tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
+ PgStat_StatTabEntry *tabentry)
+{
+ Datum values[EXTVACHEAPSTAT_COLUMNS];
+ bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_fetched -
+ tabentry->vacuum_ext.blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, tabentry->vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(tabentry->vacuum_ext.interrupts);
+
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+}
+
+/*
+ * Get the vacuum statistics for the heap tables or indexes.
+ */
+static void
+pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* Check if caller supports us returning a tuplestore */
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("set-valued function called in context that cannot accept a set")));
+ Assert(rsinfo->setDesc->natts == ncolumns);
+ Assert(rsinfo->setResult != NULL);
+
+ /* Load table statistics for specified database. */
+ if (OidIsValid(relid))
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ if (tabentry == NULL)
+ /* Table don't exists or isn't an heap relation. */
+ return;
+
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
+
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ Oid reloid;
+
+ CHECK_FOR_INTERRUPTS();
+
+ tabentry = (PgStat_StatTabEntry *) entry->data;
+ reloid = entry->key.objoid;
+
+ if (tabentry != NULL)
+ tuplestore_put_for_relation(reloid, rsinfo, tabentry);
+ }
+ }
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 5cbb5b54168..5ead2a8aff8 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ff5436acacf..8c6dbd4736a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12254,5 +12254,13 @@
proallargtypes => '{int8,pg_lsn,pg_lsn,int4}', proargmodes => '{o,o,o,o}',
proargnames => '{summarized_tli,summarized_lsn,pending_lsn,summarizer_pid}',
prosrc => 'pg_get_wal_summarizer_state' },
-
+{ oid => '8001',
+ descr => 'pg_stat_vacuum_tables return stats values',
+ proname => 'pg_stat_vacuum_tables', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_tables' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index be2c91168a1..8ab80dfe17e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,6 +169,52 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ int64 total_blks_read; /* number of pages that were missed in shared buffers during a vacuum of specific relation */
+ int64 total_blks_hit; /* number of pages that were found in shared buffers during a vacuum of specific relation */
+ int64 total_blks_dirtied; /* number of pages marked as 'Dirty' during a vacuum of specific relation. */
+ int64 total_blks_written; /* number of pages written during a vacuum of specific relation. */
+
+ int64 blks_fetched; /* number of a relation blocks, fetched during the vacuum. */
+ int64 blks_hit; /* number of a relation blocks, found in shared buffers during the vacuum. */
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double system_time; /* amount of time the CPU was busy executing vacuum code in kernel space, in msec */
+ double user_time; /* amount of time the CPU was busy executing vacuum code in user space, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 interrupts;
+
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -209,6 +255,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -267,7 +323,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAE
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
typedef struct PgStat_ArchiverStats
{
@@ -386,6 +442,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -459,6 +517,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -624,10 +687,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -675,6 +740,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -692,7 +768,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
/*
* Functions in pgstat_replslot.c
*/
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index e54eca5b489..e752c0ce015 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index fb132e439dc..24ab3ceb717 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -549,7 +549,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind, Oid dboid,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, Oid objoid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..7cdb79c0ec4
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 100| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 100| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..e93dd4f626c 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..7d31ddbece9
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,51 @@
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
+# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 0d734169f11..9ae743eae0c 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,9 +32,10 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
------+---------
-(0 rows)
+ oid | proname
+------+-----------------------
+ 8001 | pg_stat_vacuum_tables
+(1 row)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a1626f3fae9..10a7a6a6870 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2235,6 +2235,40 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..1a7d04b0590
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,200 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | t | t
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+UPDATE vestat SET x = x+1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | f | f
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 7a5a910562e..20640cd72f4 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..41e387dd304
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,158 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+UPDATE vestat SET x = x+1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v7-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (40.6K, 3-v7-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From d737134c8c82c2059577f47eebf1f142efc18e58 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:09:21 +0300
Subject: [PATCH 2/3] Machinery for grabbing an extended vacuum statistics on
heap and index relations. Remember, statistic on heap and index relations a
bit different (see ExtVacReport to find out more information). The concept of
the ExtVacReport structure has been complicated to store statistic
information for two kinds of relations: for heap and index relations.
ExtVacReportType variable helps to determine what the kind is considering
now.
---
src/backend/access/heap/vacuumlazy.c | 99 +++++++++--
src/backend/catalog/system_views.sql | 41 +++++
src/backend/utils/activity/pgstat.c | 7 +-
src/backend/utils/activity/pgstat_relation.c | 41 +++--
src/backend/utils/adt/pgstatfuncs.c | 99 ++++++-----
src/include/catalog/pg_proc.dat | 9 +
src/include/pgstat.h | 52 ++++--
.../vacuum-extending-in-repetable-read.out | 7 +-
.../vacuum-extending-in-repetable-read.spec | 2 +-
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/expected/rules.out | 26 +++
.../expected/vacuum_index_statistics.out | 158 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 3 +-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 128 ++++++++++++++
15 files changed, 599 insertions(+), 81 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d63303c7fb7..9c53d0b4c57 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -168,6 +168,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -246,6 +247,13 @@ typedef struct LVExtStatCounters
PgStat_Counter blocks_hit;
} LVExtStatCounters;
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -408,6 +416,46 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+static void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+static void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -711,14 +759,15 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(rel, &extVacCounters, &extVacReport);
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.pages_frozen = vacrel->set_frozen_pages;
- extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.type = PGSTAT_EXTVAC_HEAP;
+ extVacReport.heap.pages_scanned = vacrel->scanned_pages;
+ extVacReport.heap.pages_removed = vacrel->removed_pages;
+ extVacReport.heap.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.heap.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.heap.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.heap.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.heap.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.heap.index_vacuum_count = vacrel->num_index_scans;
/*
* Report results to the cumulative stats system, too.
@@ -2583,6 +2632,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2601,6 +2654,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2609,6 +2663,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, (void *) vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2633,6 +2694,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2652,12 +2717,20 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3274,7 +3347,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3291,7 +3364,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3307,16 +3380,22 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 31be7f04476..bbc8a430712 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1431,3 +1431,44 @@ WHERE
db.datname = current_database() AND
rel.oid = stats.relid AND
ns.oid = rel.relnamespace;
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_deleted,
+ stats.tuples_deleted,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stat_vacuum_indexes(rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace;
+
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 808a5f15c82..fb183d5b733 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1073,7 +1073,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
PG_TRY();
{
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
- pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ if (kind == PGSTAT_KIND_RELATION)
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
}
PG_FINALLY();
{
@@ -1128,6 +1129,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 791d777fbc6..5c95363c04a 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -213,7 +213,7 @@ pgstat_drop_relation(Relation rel)
* ---------
*/
void
-pgstat_report_vacuum_error(Oid tableoid)
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -230,6 +230,7 @@ pgstat_report_vacuum_error(Oid tableoid)
tabentry = &shtabentry->stats;
tabentry->vacuum_ext.interrupts++;
+ tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
}
@@ -1042,15 +1043,31 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->pages_frozen += src->pages_frozen;
- dst->pages_all_visible += src->pages_all_visible;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->dead_tuples += src->dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 0966c0bf28b..84387507ce7 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2089,17 +2089,19 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
}
#define EXTVACHEAPSTAT_COLUMNS 27
+#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
static void
tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
PgStat_StatTabEntry *tabentry)
{
- Datum values[EXTVACHEAPSTAT_COLUMNS];
- bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ Datum values[EXTVACSTAT_COLUMNS];
+ bool nulls[EXTVACSTAT_COLUMNS];
char buf[256];
int i = 0;
- memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+ memset(nulls, 0, EXTVACSTAT_COLUMNS * sizeof(bool));
values[i++] = ObjectIdGetDatum(relid);
@@ -2112,16 +2114,25 @@ tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
tabentry->vacuum_ext.blks_hit);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
- values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
- values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+ if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_HEAP)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ }
+ else if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.pages_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.tuples_deleted);
+ }
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
@@ -2149,10 +2160,9 @@ tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
* Get the vacuum statistics for the heap tables or indexes.
*/
static void
-pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
{
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
- Oid relid = PG_GETARG_OID(0);
PgStat_StatTabEntry *tabentry;
InitMaterializedSRF(fcinfo, 0);
@@ -2165,35 +2175,37 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Assert(rsinfo->setDesc->natts == ncolumns);
Assert(rsinfo->setResult != NULL);
- /* Load table statistics for specified database. */
- if (OidIsValid(relid))
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
{
- tabentry = pgstat_fetch_stat_tabentry(relid);
- if (tabentry == NULL)
- /* Table don't exists or isn't an heap relation. */
- return;
+ Oid relid = PG_GETARG_OID(0);
- tuplestore_put_for_relation(relid, rsinfo, tabentry);
- }
- else
- {
- SnapshotIterator hashiter;
- PgStat_SnapshotEntry *entry;
-
- /* Iterate the snapshot */
- InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+ /* Load table statistics for specified relation. */
+ if (OidIsValid(relid))
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ if (tabentry == NULL || tabentry->vacuum_ext.type != type)
+ /* Table don't exists or isn't an heap relation. */
+ return;
- while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
+ else
{
- Oid reloid;
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
+
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
- CHECK_FOR_INTERRUPTS();
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
- tabentry = (PgStat_StatTabEntry *) entry->data;
- reloid = entry->key.objoid;
+ tabentry = (PgStat_StatTabEntry *) entry->data;
- if (tabentry != NULL)
- tuplestore_put_for_relation(reloid, rsinfo, tabentry);
+ if (tabentry != NULL && tabentry->vacuum_ext.type == type)
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
}
}
}
@@ -2204,7 +2216,18 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Datum
pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
{
- pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
PG_RETURN_VOID();
}
+
+/*
+ * Get the vacuum statistics for the indexes.
+ */
+Datum
+pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+ }
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8c6dbd4736a..c33419552fe 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12263,4 +12263,13 @@
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
prosrc => 'pg_stat_vacuum_tables' },
+{ oid => '8002',
+ descr => 'pg_stat_vacuum_indexes return stats values',
+ proname => 'pg_stat_vacuum_indexes', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_indexes' }
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 8ab80dfe17e..2e99befe5d0 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,11 +169,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_HEAP = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -205,14 +213,38 @@ typedef struct ExtVacReport
/* Interruptions on any errors. */
int32 interrupts;
- int64 pages_scanned; /* number of pages we examined */
- int64 pages_removed; /* number of pages removed by vacuum */
- int64 pages_frozen; /* number of pages marked in VM as frozen */
- int64 pages_all_visible; /* number of pages marked in VM as all-visible */
- int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
- int64 index_vacuum_count; /* number of index vacuumings */
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } heap;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -692,7 +724,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 7cdb79c0ec4..93fe15c01f9 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -9,10 +9,9 @@ step s2_print_vacuum_stats_table:
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
-relname |tuples_deleted|dead_tuples|tuples_frozen
---------------------------+--------------+-----------+-------------
-test_vacuum_stat_isolation| 0| 0| 0
-(1 row)
+relname|tuples_deleted|dead_tuples|tuples_frozen
+-------+--------------+-----------+-------------
+(0 rows)
step s1_begin_repeatable_read:
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 7d31ddbece9..bca3e8516b2 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -48,4 +48,4 @@ permutation
s1_commit
s2_checkpoint
s2_vacuum
- s2_print_vacuum_stats_table
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 9ae743eae0c..5d72b970b03 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,10 +32,11 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
-------+-----------------------
+ oid | proname
+------+------------------------
8001 | pg_stat_vacuum_tables
-(1 row)
+ 8002 | pg_stat_vacuum_indexes
+(2 rows)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 10a7a6a6870..f39a9f6e5a0 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2235,6 +2235,32 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stat_vacuum_indexes(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..a0da8d25f1a
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,158 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+---------+----------+---------------+----------------
+(0 rows)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index 1a7d04b0590..b85a5cab9af 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -37,8 +37,7 @@ FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
---------+--------------+----------------+----------+---------------+---------------
- vestat | 0 | 0 | 455 | 0 | 0
-(1 row)
+(0 rows)
SELECT relpages AS rp
FROM pg_class c
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 20640cd72f4..25754ff6bd1 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..9113fd26e6f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,128 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] v7-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (19.9K, 4-v7-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 23a6233f182ea1dea7a9dfa5b3984d17eb7bb3b6 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:42:28 +0300
Subject: [PATCH 3/3] Machinery for grabbing an extended vacuum statistics on
databases. It transmits vacuum statistical information about each table and
accumulates it for the database which the table belonged.
---
src/backend/catalog/system_views.sql | 28 +++++++
src/backend/utils/activity/pgstat.c | 2 +
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 16 ++++
src/backend/utils/adt/pgstatfuncs.c | 75 +++++++++++++++++-
src/include/catalog/pg_proc.dat | 11 ++-
src/include/pgstat.h | 3 +-
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/expected/rules.out | 20 ++++-
...ut => vacuum_tables_and_db_statistics.out} | 78 +++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 66 +++++++++++++++-
12 files changed, 300 insertions(+), 9 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (76%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (78%)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index bbc8a430712..f8cee5e79c4 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1472,3 +1472,31 @@ WHERE
rel.oid = stats.relid AND
ns.oid = rel.relnamespace;
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+
+ stats.interrupts
+FROM
+ pg_database db LEFT JOIN pg_stat_vacuum_database(db.oid) stats
+ON
+ db.oid = stats.dboid;
+
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index fb183d5b733..583c3ff0f03 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1075,6 +1075,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
if (kind == PGSTAT_KIND_RELATION)
pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ else if (kind == PGSTAT_KIND_DATABASE)
+ pgstat_build_snapshot(PGSTAT_KIND_DATABASE);
}
PG_FINALLY();
{
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 29bc0909748..a060d1a4042 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -430,6 +430,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 5c95363c04a..725e26423f2 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -219,6 +219,7 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
if (!pgstat_track_counts)
return;
@@ -232,6 +233,10 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
tabentry->vacuum_ext.interrupts++;
tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.interrupts++;
+ dbentry->vacuum_ext.type = m_type;
}
/*
@@ -245,6 +250,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -298,6 +304,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 84387507ce7..11820a5791c 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2090,8 +2090,49 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
#define EXTVACHEAPSTAT_COLUMNS 27
#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACDBSTAT_COLUMNS 15
#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
+static void
+tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
+ PgStatShared_Database *dbentry)
+{
+ Datum values[EXTVACDBSTAT_COLUMNS];
+ bool nulls[EXTVACDBSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACDBSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->stats.vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(dbentry->stats.vacuum_ext.interrupts);
+
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+}
+
static void
tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
PgStat_StatTabEntry *tabentry)
@@ -2208,6 +2249,26 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
}
}
}
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ PgStatShared_Database *dbentry;
+ PgStat_EntryRef *entry_ref;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (OidIsValid(dbid))
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dbid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ if (dbentry == NULL)
+ /* Table doesn't exist or isn't a heap relation */
+ return;
+
+ tuplestore_put_for_database(dbid, rsinfo, dbentry);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
}
/*
@@ -2230,4 +2291,16 @@ pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
PG_RETURN_VOID();
- }
\ No newline at end of file
+ }
+
+
+/*
+ * Get the vacuum statistics for the database.
+ */
+Datum
+pg_stat_vacuum_database(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c33419552fe..b04711bb0a3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12271,5 +12271,14 @@
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
- prosrc => 'pg_stat_vacuum_indexes' }
+ prosrc => 'pg_stat_vacuum_indexes' },
+{ oid => '8003',
+ descr => 'pg_stat_vacuum_database return stats values',
+ proname => 'pg_stat_vacuum_database', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 2e99befe5d0..4c8b0a45331 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -174,7 +174,8 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_HEAP = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 5d72b970b03..7026de157e4 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,11 +32,12 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
-------+------------------------
+ oid | proname
+------+-------------------------
8001 | pg_stat_vacuum_tables
8002 | pg_stat_vacuum_indexes
-(2 rows)
+ 8003 | pg_stat_vacuum_database
+(3 rows)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f39a9f6e5a0..bedcae46fc7 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2235,6 +2235,24 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM (pg_database db
+ LEFT JOIN LATERAL pg_stat_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON ((db.oid = stats.dboid)));
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
@@ -2259,7 +2277,7 @@ pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
FROM pg_database db,
pg_class rel,
pg_namespace ns,
- LATERAL pg_stat_vacuum_indexes(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 76%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index b85a5cab9af..ec0cf97e2da 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,6 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -196,4 +199,79 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
t | t | t | t
(1 row)
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 25754ff6bd1..301be04a3d6 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 78%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 41e387dd304..ed9bb852625 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,6 +7,10 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+
-- conditio sine qua non
SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
@@ -155,4 +159,64 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+DROP TABLE vestat CASCADE;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
--
2.34.1
[text/x-patch] v7-0004-Add-documentation-about-the-system-views-that-are-us.patch (24.2K, 5-v7-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 68988e25deb68a944dc3620a13360172e23bca68 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:47:55 +0300
Subject: [PATCH 4/4] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 747 +++++++++++++++++++++++++++++++++
1 file changed, 747 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 634a4c0fab4..8cbccdc4a4d 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5064,4 +5064,751 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stats-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this index
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-frozen in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_all_visible</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-visible in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this table
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
+
</chapter>
--
2.34.1
[text/plain] minor-vacuum.no-cbot (7.9K, 6-minor-vacuum.no-cbot)
download | inline diff:
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 4e2ae78d255..9c53d0b4c57 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -3346,7 +3346,7 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
- if(geterrelevel() >= ERROR)
+ if(geterrelevel() == ERROR)
pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -3363,7 +3363,7 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
- if(geterrelevel() >= ERROR)
+ if(geterrelevel() == ERROR)
pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -3380,21 +3380,21 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
- if(geterrelevel() >= ERROR)
+ if(geterrelevel() == ERROR)
pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
- if(geterrelevel() >= ERROR)
+ if(geterrelevel() == ERROR)
pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
- if(geterrelevel() >= ERROR)
+ if(geterrelevel() == ERROR)
pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index b633408777e..583c3ff0f03 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -829,57 +829,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info)
-{
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->system_time += src->system_time;
- dst->user_time += src->user_time;
- dst->total_time += src->total_time;
- dst->interrupts += src->interrupts;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_HEAP)
- {
- dst->heap.pages_scanned += src->heap.pages_scanned;
- dst->heap.pages_removed += src->heap.pages_removed;
- dst->heap.pages_frozen += src->heap.pages_frozen;
- dst->heap.pages_all_visible += src->heap.pages_all_visible;
- dst->heap.tuples_deleted += src->heap.tuples_deleted;
- dst->heap.tuples_frozen += src->heap.tuples_frozen;
- dst->heap.dead_tuples += src->heap.dead_tuples;
- dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->index.tuples_deleted += src->index.tuples_deleted;
- }
- }
-}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index cc09aba571f..e05de63b2f0 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -48,6 +48,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -1034,3 +1036,66 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->system_time += src->system_time;
+ dst->user_time += src->user_time;
+ dst->total_time += src->total_time;
+ dst->interrupts += src->interrupts;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->dead_tuples += src->dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
+}
\ No newline at end of file
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index c56c54de3b4..eacbee579b3 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -646,7 +646,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -721,9 +721,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-extern void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
* Functions in pgstat_replslot.c
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-09-05 12:47 jian he <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: jian he @ 2024-09-05 12:47 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
On Thu, Sep 5, 2024 at 1:23 AM Alena Rybakina <[email protected]> wrote:
>
> Hi, all!
>
> I have attached the new version of the code and the diff files
> (minor-vacuum.no-cbot).
>
hi.
still have white space issue when using "git apply",
you may need to use "git diff --check" to find out where.
/* ----------
diff --git a/src/test/regress/expected/opr_sanity.out
b/src/test/regress/expected/opr_sanity.out
index 5d72b970b03..7026de157e4 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,11 +32,12 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
-------+------------------------
+ oid | proname
+------+-------------------------
8001 | pg_stat_vacuum_tables
8002 | pg_stat_vacuum_indexes
-(2 rows)
+ 8003 | pg_stat_vacuum_database
+(3 rows)
looking at src/test/regress/sql/opr_sanity.sql:
-- **************** pg_proc ****************
-- Look for illegal values in pg_proc fields.
SELECT p1.oid, p1.proname
FROM pg_proc as p1
WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
p1.pronargs < 0 OR
p1.pronargdefaults < 0 OR
p1.pronargdefaults > p1.pronargs OR
array_lower(p1.proargtypes, 1) != 0 OR
array_upper(p1.proargtypes, 1) != p1.pronargs-1 OR
0::oid = ANY (p1.proargtypes) OR
procost <= 0 OR
CASE WHEN proretset THEN prorows <= 0 ELSE prorows != 0 END OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
that means
oid | proname
------+-------------------------
8001 | pg_stat_vacuum_tables
8002 | pg_stat_vacuum_indexes
8003 | pg_stat_vacuum_database
These above functions, pg_proc.prorows should > 0 when
pg_proc.proretset is true.
I think that's the opr_sanity test's intention.
so you may need to change pg_proc.dat.
BTW the doc says:
prorows float4, Estimated number of result rows (zero if not proretset)
segmentation fault cases:
select * from pg_stat_vacuum_indexes(0);
select * from pg_stat_vacuum_tables(0);
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ PgStatShared_Database *dbentry;
+ PgStat_EntryRef *entry_ref;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (OidIsValid(dbid))
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dbid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ if (dbentry == NULL)
+ /* Table doesn't exist or isn't a heap relation */
+ return;
+
+ tuplestore_put_for_database(dbid, rsinfo, dbentry);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
didn't error out when dbid is invalid?
pg_stat_vacuum_tables
pg_stat_vacuum_indexes
pg_stat_vacuum_database
these functions didn't verify the only input argument oid's kind.
for example:
create table s(a int primary key) with (autovacuum_enabled = off);
create view sv as select * from s;
vacuum s;
select * from pg_stat_vacuum_tables('sv'::regclass::oid);
select * from pg_stat_vacuum_indexes('sv'::regclass::oid);
select * from pg_stat_vacuum_database('sv'::regclass::oid);
above all these 3 examples should error out? because sv is view.
in src/backend/catalog/system_views.sql
for view creation of pg_stat_vacuum_indexes
you can change to
WHERE
db.datname = current_database() AND
rel.oid = stats.relid AND
ns.oid = rel.relnamespace
AND rel.relkind = 'i':
pg_stat_vacuum_tables in in src/backend/catalog/system_views.sql
you can change to
WHERE
db.datname = current_database() AND
rel.oid = stats.relid AND
ns.oid = rel.relnamespace
AND rel.relkind = 'r':
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-09-05 21:00 Alena Rybakina <[email protected]>
parent: jian he <[email protected]>
0 siblings, 2 replies; 77+ messages in thread
From: Alena Rybakina @ 2024-09-05 21:00 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
Hi! Thank you for your review!
On 05.09.2024 15:47, jian he wrote:
> On Thu, Sep 5, 2024 at 1:23 AM Alena Rybakina<[email protected]> wrote:
>> Hi, all!
>>
>> I have attached the new version of the code and the diff files
>> (minor-vacuum.no-cbot).
>>
> hi.
>
> still have white space issue when using "git apply",
> you may need to use "git diff --check" to find out where.
>
>
> /* ----------
> diff --git a/src/test/regress/expected/opr_sanity.out
> b/src/test/regress/expected/opr_sanity.out
> index 5d72b970b03..7026de157e4 100644
> --- a/src/test/regress/expected/opr_sanity.out
> +++ b/src/test/regress/expected/opr_sanity.out
> @@ -32,11 +32,12 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
> prokind NOT IN ('f', 'a', 'w', 'p') OR
> provolatile NOT IN ('i', 's', 'v') OR
> proparallel NOT IN ('s', 'r', 'u');
> - oid | proname
> -------+------------------------
> + oid | proname
> +------+-------------------------
> 8001 | pg_stat_vacuum_tables
> 8002 | pg_stat_vacuum_indexes
> -(2 rows)
> + 8003 | pg_stat_vacuum_database
> +(3 rows)
>
>
> looking at src/test/regress/sql/opr_sanity.sql:
>
> -- **************** pg_proc ****************
> -- Look for illegal values in pg_proc fields.
>
> SELECT p1.oid, p1.proname
> FROM pg_proc as p1
> WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
> p1.pronargs < 0 OR
> p1.pronargdefaults < 0 OR
> p1.pronargdefaults > p1.pronargs OR
> array_lower(p1.proargtypes, 1) != 0 OR
> array_upper(p1.proargtypes, 1) != p1.pronargs-1 OR
> 0::oid = ANY (p1.proargtypes) OR
> procost <= 0 OR
> CASE WHEN proretset THEN prorows <= 0 ELSE prorows != 0 END OR
> prokind NOT IN ('f', 'a', 'w', 'p') OR
> provolatile NOT IN ('i', 's', 'v') OR
> proparallel NOT IN ('s', 'r', 'u');
>
> that means
> oid | proname
> ------+-------------------------
> 8001 | pg_stat_vacuum_tables
> 8002 | pg_stat_vacuum_indexes
> 8003 | pg_stat_vacuum_database
>
>
> These above functions, pg_proc.prorows should > 0 when
> pg_proc.proretset is true.
> I think that's the opr_sanity test's intention.
> so you may need to change pg_proc.dat.
>
> BTW the doc says:
> prorows float4, Estimated number of result rows (zero if not proretset)
>
I agree with you and have fixed it.
> segmentation fault cases:
> select * from pg_stat_vacuum_indexes(0);
> select * from pg_stat_vacuum_tables(0);
>
>
> + else if (type == PGSTAT_EXTVAC_DB)
> + {
> + PgStatShared_Database *dbentry;
> + PgStat_EntryRef *entry_ref;
> + Oid dbid = PG_GETARG_OID(0);
> +
> + if (OidIsValid(dbid))
> + {
> + entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
> + dbid, InvalidOid, false);
> + dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
> +
> + if (dbentry == NULL)
> + /* Table doesn't exist or isn't a heap relation */
> + return;
> +
> + tuplestore_put_for_database(dbid, rsinfo, dbentry);
> + pgstat_unlock_entry(entry_ref);
> + }
> + }
> didn't error out when dbid is invalid?
>
It is caused by the empty statistic snapshot. I have fixed that by
updating the snapshot (pgstat_update_snapshot(PGSTAT_KIND_RELATION)
function).
I also added the test to check it.
> pg_stat_vacuum_tables
> pg_stat_vacuum_indexes
> pg_stat_vacuum_database
> these functions didn't verify the only input argument oid's kind.
> for example:
>
> create table s(a int primary key) with (autovacuum_enabled = off);
> create view sv as select * from s;
> vacuum s;
> select * from pg_stat_vacuum_tables('sv'::regclass::oid);
> select * from pg_stat_vacuum_indexes('sv'::regclass::oid);
> select * from pg_stat_vacuum_database('sv'::regclass::oid);
>
> above all these 3 examples should error out? because sv is view.
I don't think so. I noticed that if we try to find the object from the
system table with the different type the Postgres returns empty rows. I
think we should do the same.
> in src/backend/catalog/system_views.sql
> for view creation of pg_stat_vacuum_indexes
> you can change to
>
> WHERE
> db.datname = current_database() AND
> rel.oid = stats.relid AND
> ns.oid = rel.relnamespace
> AND rel.relkind = 'i':
>
>
>
> pg_stat_vacuum_tables in in src/backend/catalog/system_views.sql
> you can change to
>
> WHERE
> db.datname = current_database() AND
> rel.oid = stats.relid AND
> ns.oid = rel.relnamespace
> AND rel.relkind = 'r':
>
I agree with your proposal and fixed it like that.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 11820a5791c..5406102dcbf 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2235,6 +2235,8 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
SnapshotIterator hashiter;
PgStat_SnapshotEntry *entry;
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+
/* Iterate the snapshot */
InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
@@ -2245,7 +2247,7 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
tabentry = (PgStat_StatTabEntry *) entry->data;
if (tabentry != NULL && tabentry->vacuum_ext.type == type)
- tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ tuplestore_put_for_relation(entry->key.objoid, rsinfo, tabentry);
}
}
}
@@ -2290,10 +2292,9 @@ pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
{
pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
- PG_RETURN_VOID();
+ PG_RETURN_VOID();
}
-
/*
* Get the vacuum statistics for the database.
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b04711bb0a3..8709a218145 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12256,7 +12256,7 @@
prosrc => 'pg_get_wal_summarizer_state' },
{ oid => '8001',
descr => 'pg_stat_vacuum_tables return stats values',
- proname => 'pg_stat_vacuum_tables', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proname => 'pg_stat_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
@@ -12265,7 +12265,7 @@
prosrc => 'pg_stat_vacuum_tables' },
{ oid => '8002',
descr => 'pg_stat_vacuum_indexes return stats values',
- proname => 'pg_stat_vacuum_indexes', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proname => 'pg_stat_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
@@ -12274,7 +12274,7 @@
prosrc => 'pg_stat_vacuum_indexes' },
{ oid => '8003',
descr => 'pg_stat_vacuum_database return stats values',
- proname => 'pg_stat_vacuum_database', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proname => 'pg_stat_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 7026de157e4..0d734169f11 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,12 +32,9 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
-------+-------------------------
- 8001 | pg_stat_vacuum_tables
- 8002 | pg_stat_vacuum_indexes
- 8003 | pg_stat_vacuum_database
-(3 rows)
+ oid | proname
+-----+---------
+(0 rows)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index bedcae46fc7..e0dcc513972 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2278,7 +2278,7 @@ pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
pg_class rel,
pg_namespace ns,
LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
- WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'i'::"char"));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
@@ -2312,7 +2312,7 @@ pg_stat_vacuum_tables| SELECT rel.oid AS relid,
pg_class rel,
pg_namespace ns,
LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
- WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'r'::"char"));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index a0da8d25f1a..4f6e305710e 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -155,4 +155,10 @@ WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
vestat_pkey | f | t | t
(1 row)
+SELECT min(relid) FROM pg_stat_vacuum_indexes(0);
+ min
+------
+ 1232
+(1 row)
+
DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_and_db_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
index ec0cf97e2da..fbbb26560df 100644
--- a/src/test/regress/expected/vacuum_tables_and_db_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -199,6 +199,12 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
t | t | t | t
(1 row)
+SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
+ min
+------
+ 1213
+(1 row)
+
-- Now check vacuum statistics for current database
SELECT dbname,
db_blks_hit > 0 AS db_blks_hit,
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index 9113fd26e6f..75e5974eb59 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -125,4 +125,6 @@ SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_
FROM pg_stat_vacuum_indexes vt, pg_class c
WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT min(relid) FROM pg_stat_vacuum_indexes(0);
+
DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index ed9bb852625..3f19936ca61 100644
--- a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -159,6 +159,8 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
+
-- Now check vacuum statistics for current database
SELECT dbname,
db_blks_hit > 0 AS db_blks_hit,
Attachments:
[text/x-patch] v8-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (63.6K, 3-v8-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From cf3f0f49a625f102c46ef641f84cce9d7afeb655 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Wed, 4 Sep 2024 18:52:40 +0300
Subject: [PATCH 1/3] Machinery for grabbing an extended vacuum statistics on
heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 159 +++++++++++++-
src/backend/access/heap/visibilitymap.c | 13 ++
src/backend/catalog/system_views.sql | 55 +++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 32 ++-
src/backend/utils/activity/pgstat_relation.c | 72 +++++-
src/backend/utils/adt/pgstatfuncs.c | 156 +++++++++++++
src/backend/utils/error/elog.c | 13 ++
src/include/catalog/pg_proc.dat | 10 +-
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 81 ++++++-
src/include/utils/elog.h | 1 +
src/include/utils/pgstat_internal.h | 2 +-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 51 +++++
src/test/regress/expected/rules.out | 34 +++
.../expected/vacuum_tables_statistics.out | 206 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 160 ++++++++++++++
21 files changed, 1096 insertions(+), 14 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d82aa3d4896..d63303c7fb7 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -167,6 +167,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -194,6 +195,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -226,6 +229,22 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Cut-off values of parameters which changes implicitly during a vacuum
+ * process.
+ * Vacuum can't control their values, so we should store them before and after
+ * the processing.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz time;
+ PGRUsage ru;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -279,6 +298,115 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+ PGRUsage ru0;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ pg_rusage_init(&ru0);
+ starttime = GetCurrentTimestamp();
+
+ counters->ru = ru0;
+ counters->time = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+ PGRUsage ru1;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->time, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ /*
+ * Get difference of a system time and user time values in milliseconds.
+ * Use floating point representation to show tails of time diffs.
+ */
+ pg_rusage_init(&ru1);
+ report->system_time =
+ (ru1.ru.ru_stime.tv_sec - counters->ru.ru.ru_stime.tv_sec) * 1000. +
+ (ru1.ru.ru_stime.tv_usec - counters->ru.ru.ru_stime.tv_usec) * 0.001;
+ report->user_time =
+ (ru1.ru.ru_utime.tv_sec - counters->ru.ru.ru_utime.tv_sec) * 1000. +
+ (ru1.ru.ru_utime.tv_usec - counters->ru.ru.ru_utime.tv_usec) * 0.001;
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -311,6 +439,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -329,7 +459,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -346,6 +476,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -413,6 +544,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -574,6 +707,19 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -588,7 +734,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1380,6 +1527,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2277,11 +2426,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -3122,6 +3273,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3137,6 +3290,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33c..d72cade60a4 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,18 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Initially, it didn't matter what type of flags (all-visible or frozen) we received,
+ * we just performed a reverse concatenation operation. But this information is very important
+ * for vacuum statistics. We need to find out this usingthe bit concatenation operation
+ * with the VISIBILITYMAP_ALL_VISIBLE and VISIBILITYMAP_ALL_FROZEN masks,
+ * and where the desired one matches, we increment the value there.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 7fd5d256a18..247147e0213 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1377,3 +1377,58 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stat_vacuum_tables(rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace AND
+ rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 7d8e9d20454..363924d00db 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -103,6 +103,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2394,6 +2397,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 22c057fe61b..13ab633086a 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1043,6 +1043,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 178b5ef65aa..0ae367585fb 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -260,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -840,7 +839,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -906,7 +904,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, Oid objoid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1022,7 +1020,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1072,8 +1070,30 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 8a3f7d434cf..791d777fbc6 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -48,6 +48,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -204,12 +206,40 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.interrupts++;
+ pgstat_unlock_entry(entry_ref);
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -233,6 +263,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -861,6 +893,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -984,3 +1019,38 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->system_time += src->system_time;
+ dst->user_time += src->user_time;
+ dst->total_time += src->total_time;
+ dst->interrupts += src->interrupts;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->dead_tuples += src->dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 97dc09ac0d9..86ba2a68bae 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -31,6 +31,42 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+#include "utils/pgstat_internal.h"
+
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -2051,3 +2087,123 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objoid));
}
+
+#define EXTVACHEAPSTAT_COLUMNS 27
+
+static void
+tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
+ PgStat_StatTabEntry *tabentry)
+{
+ Datum values[EXTVACHEAPSTAT_COLUMNS];
+ bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_fetched -
+ tabentry->vacuum_ext.blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, tabentry->vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(tabentry->vacuum_ext.interrupts);
+
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+}
+
+/*
+ * Get the vacuum statistics for the heap tables or indexes.
+ */
+static void
+pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* Check if caller supports us returning a tuplestore */
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("set-valued function called in context that cannot accept a set")));
+ Assert(rsinfo->setDesc->natts == ncolumns);
+ Assert(rsinfo->setResult != NULL);
+
+ /* Load table statistics for specified database. */
+ if (OidIsValid(relid))
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ if (tabentry == NULL)
+ /* Table don't exists or isn't an heap relation. */
+ return;
+
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
+
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ tabentry = (PgStat_StatTabEntry *) entry->data;
+
+ if (tabentry != NULL)
+ tuplestore_put_for_relation(entry->key.objoid, rsinfo, tabentry);
+ }
+ }
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 5cbb5b54168..5ead2a8aff8 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ff5436acacf..d57e181c419 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12254,5 +12254,13 @@
proallargtypes => '{int8,pg_lsn,pg_lsn,int4}', proargmodes => '{o,o,o,o}',
proargnames => '{summarized_tli,summarized_lsn,pending_lsn,summarizer_pid}',
prosrc => 'pg_get_wal_summarizer_state' },
-
+{ oid => '8001',
+ descr => 'pg_stat_vacuum_tables return stats values',
+ proname => 'pg_stat_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_tables' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index be2c91168a1..8ab80dfe17e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,6 +169,52 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ int64 total_blks_read; /* number of pages that were missed in shared buffers during a vacuum of specific relation */
+ int64 total_blks_hit; /* number of pages that were found in shared buffers during a vacuum of specific relation */
+ int64 total_blks_dirtied; /* number of pages marked as 'Dirty' during a vacuum of specific relation. */
+ int64 total_blks_written; /* number of pages written during a vacuum of specific relation. */
+
+ int64 blks_fetched; /* number of a relation blocks, fetched during the vacuum. */
+ int64 blks_hit; /* number of a relation blocks, found in shared buffers during the vacuum. */
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double system_time; /* amount of time the CPU was busy executing vacuum code in kernel space, in msec */
+ double user_time; /* amount of time the CPU was busy executing vacuum code in user space, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 interrupts;
+
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -209,6 +255,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -267,7 +323,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAE
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
typedef struct PgStat_ArchiverStats
{
@@ -386,6 +442,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -459,6 +517,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -624,10 +687,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -675,6 +740,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -692,7 +768,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
/*
* Functions in pgstat_replslot.c
*/
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index e54eca5b489..e752c0ce015 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 25820cbf0a6..a8bba84cb6c 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -555,7 +555,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind, Oid dboid,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, Oid objoid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..7cdb79c0ec4
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 100| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 100| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..e93dd4f626c 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..7d31ddbece9
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,51 @@
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
+# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a1626f3fae9..87ce782153b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2235,6 +2235,40 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'r'::"char"));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..f89e0df79c5
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,206 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | t | t
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+UPDATE vestat SET x = x+1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | f | f
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ t | t | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
+ min
+-----
+ 112
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 7a5a910562e..20640cd72f4 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..e2e5a5794c3
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,160 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+UPDATE vestat SET x = x+1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v8-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (40.3K, 4-v8-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From d370a76665bdd5e57a42916ddb234b1ba6be906a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 5 Sep 2024 21:00:35 +0300
Subject: [PATCH 2/3] Machinery for grabbing an extended vacuum statistics on
heap and index relations. Remember, statistic on heap and index relations a
bit different (see ExtVacReport to find out more information). The concept of
the ExtVacReport structure has been complicated to store statistic
information for two kinds of relations: for heap and index relations.
ExtVacReportType variable helps to determine what the kind is considering
now.
---
src/backend/access/heap/vacuumlazy.c | 99 +++++++++--
src/backend/catalog/system_views.sql | 40 +++++
src/backend/utils/activity/pgstat.c | 7 +-
src/backend/utils/activity/pgstat_relation.c | 41 +++--
src/backend/utils/adt/pgstatfuncs.c | 100 +++++++----
src/include/catalog/pg_proc.dat | 9 +
src/include/pgstat.h | 52 ++++--
.../vacuum-extending-in-repetable-read.out | 7 +-
.../vacuum-extending-in-repetable-read.spec | 2 +-
src/test/regress/expected/rules.out | 26 +++
.../expected/vacuum_index_statistics.out | 164 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 9 +-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 130 ++++++++++++++
14 files changed, 607 insertions(+), 80 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d63303c7fb7..9c53d0b4c57 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -168,6 +168,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -246,6 +247,13 @@ typedef struct LVExtStatCounters
PgStat_Counter blocks_hit;
} LVExtStatCounters;
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -408,6 +416,46 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+static void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+static void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -711,14 +759,15 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(rel, &extVacCounters, &extVacReport);
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.pages_frozen = vacrel->set_frozen_pages;
- extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.type = PGSTAT_EXTVAC_HEAP;
+ extVacReport.heap.pages_scanned = vacrel->scanned_pages;
+ extVacReport.heap.pages_removed = vacrel->removed_pages;
+ extVacReport.heap.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.heap.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.heap.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.heap.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.heap.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.heap.index_vacuum_count = vacrel->num_index_scans;
/*
* Report results to the cumulative stats system, too.
@@ -2583,6 +2632,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2601,6 +2654,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2609,6 +2663,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, (void *) vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2633,6 +2694,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2652,12 +2717,20 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3274,7 +3347,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3291,7 +3364,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3307,16 +3380,22 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 247147e0213..92f82373c6a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1432,3 +1432,43 @@ WHERE
rel.oid = stats.relid AND
ns.oid = rel.relnamespace AND
rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_deleted,
+ stats.tuples_deleted,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stat_vacuum_indexes(rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace AND
+ rel.relkind = 'i';
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 0ae367585fb..1c2f0078880 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1083,7 +1083,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
PG_TRY();
{
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
- pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ if (kind == PGSTAT_KIND_RELATION)
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
}
PG_FINALLY();
{
@@ -1138,6 +1139,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 791d777fbc6..5c95363c04a 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -213,7 +213,7 @@ pgstat_drop_relation(Relation rel)
* ---------
*/
void
-pgstat_report_vacuum_error(Oid tableoid)
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -230,6 +230,7 @@ pgstat_report_vacuum_error(Oid tableoid)
tabentry = &shtabentry->stats;
tabentry->vacuum_ext.interrupts++;
+ tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
}
@@ -1042,15 +1043,31 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->pages_frozen += src->pages_frozen;
- dst->pages_all_visible += src->pages_all_visible;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->dead_tuples += src->dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 86ba2a68bae..b08de122674 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2089,17 +2089,19 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
}
#define EXTVACHEAPSTAT_COLUMNS 27
+#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
static void
tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
PgStat_StatTabEntry *tabentry)
{
- Datum values[EXTVACHEAPSTAT_COLUMNS];
- bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ Datum values[EXTVACSTAT_COLUMNS];
+ bool nulls[EXTVACSTAT_COLUMNS];
char buf[256];
int i = 0;
- memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+ memset(nulls, 0, EXTVACSTAT_COLUMNS * sizeof(bool));
values[i++] = ObjectIdGetDatum(relid);
@@ -2112,16 +2114,25 @@ tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
tabentry->vacuum_ext.blks_hit);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
- values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
- values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+ if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_HEAP)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ }
+ else if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.pages_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.tuples_deleted);
+ }
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
@@ -2149,10 +2160,9 @@ tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
* Get the vacuum statistics for the heap tables or indexes.
*/
static void
-pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
{
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
- Oid relid = PG_GETARG_OID(0);
PgStat_StatTabEntry *tabentry;
InitMaterializedSRF(fcinfo, 0);
@@ -2165,34 +2175,39 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Assert(rsinfo->setDesc->natts == ncolumns);
Assert(rsinfo->setResult != NULL);
- /* Load table statistics for specified database. */
- if (OidIsValid(relid))
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
{
- tabentry = pgstat_fetch_stat_tabentry(relid);
- if (tabentry == NULL)
- /* Table don't exists or isn't an heap relation. */
- return;
+ Oid relid = PG_GETARG_OID(0);
- tuplestore_put_for_relation(relid, rsinfo, tabentry);
- }
- else
- {
- SnapshotIterator hashiter;
- PgStat_SnapshotEntry *entry;
+ /* Load table statistics for specified relation. */
+ if (OidIsValid(relid))
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ if (tabentry == NULL || tabentry->vacuum_ext.type != type)
+ /* Table don't exists or isn't an heap relation. */
+ return;
- pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
- /* Iterate the snapshot */
- InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
- while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
- {
- CHECK_FOR_INTERRUPTS();
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
- tabentry = (PgStat_StatTabEntry *) entry->data;
+ tabentry = (PgStat_StatTabEntry *) entry->data;
- if (tabentry != NULL)
- tuplestore_put_for_relation(entry->key.objoid, rsinfo, tabentry);
+ if (tabentry != NULL && tabentry->vacuum_ext.type == type)
+ tuplestore_put_for_relation(entry->key.objoid, rsinfo, tabentry);
+ }
}
}
}
@@ -2203,7 +2218,18 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Datum
pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
{
- pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
PG_RETURN_VOID();
}
+
+/*
+ * Get the vacuum statistics for the indexes.
+ */
+Datum
+pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+ }
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d57e181c419..5fed40d4dc8 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12263,4 +12263,13 @@
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
prosrc => 'pg_stat_vacuum_tables' },
+{ oid => '8002',
+ descr => 'pg_stat_vacuum_indexes return stats values',
+ proname => 'pg_stat_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_indexes' }
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 8ab80dfe17e..2e99befe5d0 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,11 +169,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_HEAP = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -205,14 +213,38 @@ typedef struct ExtVacReport
/* Interruptions on any errors. */
int32 interrupts;
- int64 pages_scanned; /* number of pages we examined */
- int64 pages_removed; /* number of pages removed by vacuum */
- int64 pages_frozen; /* number of pages marked in VM as frozen */
- int64 pages_all_visible; /* number of pages marked in VM as all-visible */
- int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
- int64 index_vacuum_count; /* number of index vacuumings */
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } heap;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -692,7 +724,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 7cdb79c0ec4..93fe15c01f9 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -9,10 +9,9 @@ step s2_print_vacuum_stats_table:
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
-relname |tuples_deleted|dead_tuples|tuples_frozen
---------------------------+--------------+-----------+-------------
-test_vacuum_stat_isolation| 0| 0| 0
-(1 row)
+relname|tuples_deleted|dead_tuples|tuples_frozen
+-------+--------------+-----------+-------------
+(0 rows)
step s1_begin_repeatable_read:
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 7d31ddbece9..bca3e8516b2 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -48,4 +48,4 @@ permutation
s1_commit
s2_checkpoint
s2_vacuum
- s2_print_vacuum_stats_table
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 87ce782153b..c312428e62b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2235,6 +2235,32 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'i'::"char"));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..4f6e305710e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,164 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+---------+----------+---------------+----------------
+(0 rows)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_vacuum_indexes(0);
+ min
+------
+ 1232
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index f89e0df79c5..069ad35056c 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -37,8 +37,7 @@ FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
---------+--------------+----------------+----------+---------------+---------------
- vestat | 0 | 0 | 455 | 0 | 0
-(1 row)
+(0 rows)
SELECT relpages AS rp
FROM pg_class c
@@ -198,9 +197,9 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
(1 row)
SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
- min
------
- 112
+ min
+------
+ 1213
(1 row)
DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 20640cd72f4..25754ff6bd1 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..75e5974eb59
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,130 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+SELECT min(relid) FROM pg_stat_vacuum_indexes(0);
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] v8-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (18.0K, 5-v8-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From b70f6a8fc5d1eda1f1dd40339feee99c03a99d25 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 5 Sep 2024 20:53:05 +0300
Subject: [PATCH 3/3] Machinery for grabbing an extended vacuum statistics on
databases. It transmits vacuum statistical information about each table and
accumulates it for the database which the table belonged.
---
src/backend/catalog/system_views.sql | 28 +++++++
src/backend/utils/activity/pgstat.c | 2 +
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 16 ++++
src/backend/utils/adt/pgstatfuncs.c | 74 +++++++++++++++++-
src/include/catalog/pg_proc.dat | 11 ++-
src/include/pgstat.h | 3 +-
src/test/regress/expected/rules.out | 18 +++++
...ut => vacuum_tables_and_db_statistics.out} | 78 +++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 66 +++++++++++++++-
11 files changed, 294 insertions(+), 5 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (76%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (78%)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 92f82373c6a..d5c0ae8b37c 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1472,3 +1472,31 @@ WHERE
rel.oid = stats.relid AND
ns.oid = rel.relnamespace AND
rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+
+ stats.interrupts
+FROM
+ pg_database db LEFT JOIN pg_stat_vacuum_database(db.oid) stats
+ON
+ db.oid = stats.dboid;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 1c2f0078880..fbac089f627 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1085,6 +1085,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
if (kind == PGSTAT_KIND_RELATION)
pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ else if (kind == PGSTAT_KIND_DATABASE)
+ pgstat_build_snapshot(PGSTAT_KIND_DATABASE);
}
PG_FINALLY();
{
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 29bc0909748..a060d1a4042 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -430,6 +430,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 5c95363c04a..725e26423f2 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -219,6 +219,7 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
if (!pgstat_track_counts)
return;
@@ -232,6 +233,10 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
tabentry->vacuum_ext.interrupts++;
tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.interrupts++;
+ dbentry->vacuum_ext.type = m_type;
}
/*
@@ -245,6 +250,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -298,6 +304,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b08de122674..5406102dcbf 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2090,8 +2090,49 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
#define EXTVACHEAPSTAT_COLUMNS 27
#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACDBSTAT_COLUMNS 15
#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
+static void
+tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
+ PgStatShared_Database *dbentry)
+{
+ Datum values[EXTVACDBSTAT_COLUMNS];
+ bool nulls[EXTVACDBSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACDBSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->stats.vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(dbentry->stats.vacuum_ext.interrupts);
+
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+}
+
static void
tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
PgStat_StatTabEntry *tabentry)
@@ -2210,6 +2251,26 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
}
}
}
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ PgStatShared_Database *dbentry;
+ PgStat_EntryRef *entry_ref;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (OidIsValid(dbid))
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dbid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ if (dbentry == NULL)
+ /* Table doesn't exist or isn't a heap relation */
+ return;
+
+ tuplestore_put_for_database(dbid, rsinfo, dbentry);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
}
/*
@@ -2232,4 +2293,15 @@ pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
PG_RETURN_VOID();
- }
\ No newline at end of file
+ }
+
+/*
+ * Get the vacuum statistics for the database.
+ */
+Datum
+pg_stat_vacuum_database(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5fed40d4dc8..8709a218145 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12271,5 +12271,14 @@
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
- prosrc => 'pg_stat_vacuum_indexes' }
+ prosrc => 'pg_stat_vacuum_indexes' },
+{ oid => '8003',
+ descr => 'pg_stat_vacuum_database return stats values',
+ proname => 'pg_stat_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 2e99befe5d0..4c8b0a45331 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -174,7 +174,8 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_HEAP = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index c312428e62b..e0dcc513972 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2235,6 +2235,24 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM (pg_database db
+ LEFT JOIN LATERAL pg_stat_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON ((db.oid = stats.dboid)));
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 76%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index 069ad35056c..fbbb26560df 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,6 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -202,4 +205,79 @@ SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
1213
(1 row)
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 25754ff6bd1..301be04a3d6 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 78%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index e2e5a5794c3..3f19936ca61 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,6 +7,10 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+
-- conditio sine qua non
SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
@@ -157,4 +161,64 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+DROP TABLE vestat CASCADE;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
--
2.34.1
[text/x-patch] v8-0004-Add-documentation-about-the-system-views-that-are-us.patch (24.2K, 6-v8-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 68988e25deb68a944dc3620a13360172e23bca68 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:47:55 +0300
Subject: [PATCH 4/4] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 747 +++++++++++++++++++++++++++++++++
1 file changed, 747 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 634a4c0fab4..8cbccdc4a4d 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5064,4 +5064,751 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stats-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this index
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-frozen in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_all_visible</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-visible in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this table
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
+
</chapter>
--
2.34.1
[text/plain] minor-vacuum.no-cbot (7.9K, 7-minor-vacuum.no-cbot)
download | inline diff:
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 11820a5791c..5406102dcbf 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2235,6 +2235,8 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
SnapshotIterator hashiter;
PgStat_SnapshotEntry *entry;
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+
/* Iterate the snapshot */
InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
@@ -2245,7 +2247,7 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
tabentry = (PgStat_StatTabEntry *) entry->data;
if (tabentry != NULL && tabentry->vacuum_ext.type == type)
- tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ tuplestore_put_for_relation(entry->key.objoid, rsinfo, tabentry);
}
}
}
@@ -2290,10 +2292,9 @@ pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
{
pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
- PG_RETURN_VOID();
+ PG_RETURN_VOID();
}
-
/*
* Get the vacuum statistics for the database.
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b04711bb0a3..8709a218145 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12256,7 +12256,7 @@
prosrc => 'pg_get_wal_summarizer_state' },
{ oid => '8001',
descr => 'pg_stat_vacuum_tables return stats values',
- proname => 'pg_stat_vacuum_tables', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proname => 'pg_stat_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
@@ -12265,7 +12265,7 @@
prosrc => 'pg_stat_vacuum_tables' },
{ oid => '8002',
descr => 'pg_stat_vacuum_indexes return stats values',
- proname => 'pg_stat_vacuum_indexes', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proname => 'pg_stat_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
@@ -12274,7 +12274,7 @@
prosrc => 'pg_stat_vacuum_indexes' },
{ oid => '8003',
descr => 'pg_stat_vacuum_database return stats values',
- proname => 'pg_stat_vacuum_database', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proname => 'pg_stat_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 7026de157e4..0d734169f11 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,12 +32,9 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
-------+-------------------------
- 8001 | pg_stat_vacuum_tables
- 8002 | pg_stat_vacuum_indexes
- 8003 | pg_stat_vacuum_database
-(3 rows)
+ oid | proname
+-----+---------
+(0 rows)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index bedcae46fc7..e0dcc513972 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2278,7 +2278,7 @@ pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
pg_class rel,
pg_namespace ns,
LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
- WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'i'::"char"));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
@@ -2312,7 +2312,7 @@ pg_stat_vacuum_tables| SELECT rel.oid AS relid,
pg_class rel,
pg_namespace ns,
LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
- WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'r'::"char"));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index a0da8d25f1a..4f6e305710e 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -155,4 +155,10 @@ WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
vestat_pkey | f | t | t
(1 row)
+SELECT min(relid) FROM pg_stat_vacuum_indexes(0);
+ min
+------
+ 1232
+(1 row)
+
DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_and_db_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
index ec0cf97e2da..fbbb26560df 100644
--- a/src/test/regress/expected/vacuum_tables_and_db_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -199,6 +199,12 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
t | t | t | t
(1 row)
+SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
+ min
+------
+ 1213
+(1 row)
+
-- Now check vacuum statistics for current database
SELECT dbname,
db_blks_hit > 0 AS db_blks_hit,
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index 9113fd26e6f..75e5974eb59 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -125,4 +125,6 @@ SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_
FROM pg_stat_vacuum_indexes vt, pg_class c
WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT min(relid) FROM pg_stat_vacuum_indexes(0);
+
DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index ed9bb852625..3f19936ca61 100644
--- a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -159,6 +159,8 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
+
-- Now check vacuum statistics for current database
SELECT dbname,
db_blks_hit > 0 AS db_blks_hit,
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-09-27 18:15 Masahiko Sawada <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 2 replies; 77+ messages in thread
From: Masahiko Sawada @ 2024-09-27 18:15 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: jian he <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
Hi,
On Thu, Sep 5, 2024 at 2:01 PM Alena Rybakina <[email protected]> wrote:
>
> Hi! Thank you for your review!
>
> On 05.09.2024 15:47, jian he wrote:
>
> On Thu, Sep 5, 2024 at 1:23 AM Alena Rybakina <[email protected]> wrote:
>
> Hi, all!
>
> I have attached the new version of the code and the diff files
> (minor-vacuum.no-cbot).
Thank you for updating the patches. I've reviewed the 0001 patch and
have two comments.
I think we can split the 0001 patch into two parts: adding
pg_stat_vacuum_tables system views that shows the vacuum statistics
that we are currently collecting such as scanned_pages and
removed_pages, and another one is to add new statistics to collect
such as vacrel->set_all_visible_pages and visibility map updates.
I'm concerned that a pg_stat_vacuum_tables view has some duplicated
statistics that we already collect in different ways. For instance,
total_blks_{read,hit,dirtied,written} are already tracked at
system-level by pg_stat_io, and per-relation block I/O statistics can
be collected using pg_stat_statements. Having duplicated statistics
consumes more memory for pgstat and could confuse users if these
statistics are not consistent. I think it would be better to avoid
collecting duplicated statistics in different places.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-09-27 19:19 Melanie Plageman <[email protected]>
parent: Masahiko Sawada <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Melanie Plageman @ 2024-09-27 19:19 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Alena Rybakina <[email protected]>; jian he <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
On Fri, Sep 27, 2024 at 2:16 PM Masahiko Sawada <[email protected]> wrote:
>
> Hi,
>
> On Thu, Sep 5, 2024 at 2:01 PM Alena Rybakina <[email protected]> wrote:
> >
> > Hi! Thank you for your review!
> >
> > On 05.09.2024 15:47, jian he wrote:
> >
> > On Thu, Sep 5, 2024 at 1:23 AM Alena Rybakina <[email protected]> wrote:
> >
> > Hi, all!
> >
> > I have attached the new version of the code and the diff files
> > (minor-vacuum.no-cbot).
>
> Thank you for updating the patches. I've reviewed the 0001 patch and
> have two comments.
I took a very brief look at this and was wondering if it was worth
having a way to make the per-table vacuum statistics opt-in (like a
table storage parameter) in order to decrease the shared memory
footprint of storing the stats.
- Melanie
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-09-27 19:25 Andrei Zubkov <[email protected]>
parent: Masahiko Sawada <[email protected]>
1 sibling, 0 replies; 77+ messages in thread
From: Andrei Zubkov @ 2024-09-27 19:25 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; Alena Rybakina <[email protected]>; +Cc: jian he <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; pgsql-hackers; [email protected]
Hi,
On Fri, 2024-09-27 at 11:15 -0700, Masahiko Sawada wrote:
> I'm concerned that a pg_stat_vacuum_tables view has some duplicated
> statistics that we already collect in different ways. For instance,
> total_blks_{read,hit,dirtied,written} are already tracked at
> system-level by pg_stat_io,
pg_stat_vacuum_tables.total_blks_{read,hit,dirtied,written} tracks
blocks used by vacuum in different ways while vacuuming this particular
table while pg_stat_io tracks blocks used by vacuum on the cluster
level.
> and per-relation block I/O statistics can
> be collected using pg_stat_statements.
This is impossible. pg_stat_statements tracks block statistics on a
statement level. One statement could touch many tables and many
indexes, and all used database blocks will be counted by the
pg_stat_statements counters on a statement-level. Autovacuum statistics
won't be accounted by the pg_stat_statements. After all,
pg_stat_statements won't hold the statements statistics forever. Under
pressure of new statements the statement eviction can happen and
statistics will be lost.
All of the above is addressed by relation-level vacuum statistics held
in the Cumulative Statistics System proposed by this patch.
--
regards, Andrei Zubkov
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-09-27 20:13 Masahiko Sawada <[email protected]>
parent: Melanie Plageman <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Masahiko Sawada @ 2024-09-27 20:13 UTC (permalink / raw)
To: Melanie Plageman <[email protected]>; +Cc: Alena Rybakina <[email protected]>; jian he <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
On Fri, Sep 27, 2024 at 12:19 PM Melanie Plageman
<[email protected]> wrote:
>
> On Fri, Sep 27, 2024 at 2:16 PM Masahiko Sawada <[email protected]> wrote:
> >
> > Hi,
> >
> > On Thu, Sep 5, 2024 at 2:01 PM Alena Rybakina <[email protected]> wrote:
> > >
> > > Hi! Thank you for your review!
> > >
> > > On 05.09.2024 15:47, jian he wrote:
> > >
> > > On Thu, Sep 5, 2024 at 1:23 AM Alena Rybakina <[email protected]> wrote:
> > >
> > > Hi, all!
> > >
> > > I have attached the new version of the code and the diff files
> > > (minor-vacuum.no-cbot).
> >
> > Thank you for updating the patches. I've reviewed the 0001 patch and
> > have two comments.
>
> I took a very brief look at this and was wondering if it was worth
> having a way to make the per-table vacuum statistics opt-in (like a
> table storage parameter) in order to decrease the shared memory
> footprint of storing the stats.
I'm not sure how users can select tables that enable vacuum statistics
as I think they basically want to have statistics for all tables, but
I see your point. Since the size of PgStat_TableCounts approximately
tripled by this patch (112 bytes to 320 bytes), it might be worth
considering ways to reduce the number of entries or reducing the size
of vacuum statistics.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-09-28 21:22 Alena Rybakina <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2024-09-28 21:22 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; Andrei Zubkov <[email protected]>; +Cc: jian he <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
Hi! Thank you for your interesting for this patch!
>> I took a very brief look at this and was wondering if it was worth
>> having a way to make the per-table vacuum statistics opt-in (like a
>> table storage parameter) in order to decrease the shared memory
>> footprint of storing the stats.
> I'm not sure how users can select tables that enable vacuum statistics
> as I think they basically want to have statistics for all tables, but
> I see your point. Since the size of PgStat_TableCounts approximately
> tripled by this patch (112 bytes to 320 bytes), it might be worth
> considering ways to reduce the number of entries or reducing the size
> of vacuum statistics.
The main purpose of these statistics is to see abnormal behavior of
vacuum in relation to a table or the database as a whole.
For example, there may be a situation where vacuum has started to run
more often and spends a lot of resources on processing a certain index,
but the size of the index does not change significantly. Moreover, the
table in which this index is located can be much smaller in size. This
may be because the index is bloated and needs to be reindexed.
This is exactly what vacuum statistics can show - we will see that
compared to other objects, vacuum processed more blocks and spent more
time on this index.
Perhaps the vacuum parameters for the index should be set more
aggressively to avoid this in the future.
I suppose that if we turn off statistics collection for a certain
object, we can miss it. In addition, the user may not enable the
parameter for the object in time, because he will forget about it.
As for the second option, now I cannot say which statistics can be
removed, to be honest. So far, they all seem necessary.
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-10-08 16:18 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2024-10-08 16:18 UTC (permalink / raw)
To: pgsql-hackers; +Cc: jian he <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; [email protected]
Made a rebase on a fresh master branch.
--
Regards,
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] v9-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (63.6K, 3-v9-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From e4f31042cf87bf6a22df87f795901506a1c21192 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 8 Oct 2024 18:32:54 +0300
Subject: [PATCH 1/4] Machinery for grabbing an extended vacuum statistics on
heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 159 ++++++++++++-
src/backend/access/heap/visibilitymap.c | 13 ++
src/backend/catalog/system_views.sql | 55 +++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 32 ++-
src/backend/utils/activity/pgstat_relation.c | 72 +++++-
src/backend/utils/adt/pgstatfuncs.c | 156 +++++++++++++
src/backend/utils/error/elog.c | 13 ++
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 81 ++++++-
src/include/utils/elog.h | 1 +
src/include/utils/pgstat_internal.h | 2 +-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 51 +++++
src/test/regress/expected/rules.out | 34 +++
.../expected/vacuum_tables_statistics.out | 209 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 160 ++++++++++++++
21 files changed, 1099 insertions(+), 13 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d82aa3d4896..d63303c7fb7 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -167,6 +167,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -194,6 +195,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -226,6 +229,22 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Cut-off values of parameters which changes implicitly during a vacuum
+ * process.
+ * Vacuum can't control their values, so we should store them before and after
+ * the processing.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz time;
+ PGRUsage ru;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -279,6 +298,115 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+ PGRUsage ru0;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ pg_rusage_init(&ru0);
+ starttime = GetCurrentTimestamp();
+
+ counters->ru = ru0;
+ counters->time = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+ PGRUsage ru1;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->time, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ /*
+ * Get difference of a system time and user time values in milliseconds.
+ * Use floating point representation to show tails of time diffs.
+ */
+ pg_rusage_init(&ru1);
+ report->system_time =
+ (ru1.ru.ru_stime.tv_sec - counters->ru.ru.ru_stime.tv_sec) * 1000. +
+ (ru1.ru.ru_stime.tv_usec - counters->ru.ru.ru_stime.tv_usec) * 0.001;
+ report->user_time =
+ (ru1.ru.ru_utime.tv_sec - counters->ru.ru.ru_utime.tv_sec) * 1000. +
+ (ru1.ru.ru_utime.tv_usec - counters->ru.ru.ru_utime.tv_usec) * 0.001;
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -311,6 +439,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -329,7 +459,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -346,6 +476,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -413,6 +544,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -574,6 +707,19 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -588,7 +734,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1380,6 +1527,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2277,11 +2426,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -3122,6 +3273,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3137,6 +3290,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33c..d72cade60a4 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,18 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Initially, it didn't matter what type of flags (all-visible or frozen) we received,
+ * we just performed a reverse concatenation operation. But this information is very important
+ * for vacuum statistics. We need to find out this usingthe bit concatenation operation
+ * with the VISIBILITYMAP_ALL_VISIBLE and VISIBILITYMAP_ALL_FROZEN masks,
+ * and where the desired one matches, we increment the value there.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 3456b821bc5..ec997531326 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1379,3 +1379,58 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stat_vacuum_tables(rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace AND
+ rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ac8f5d9c259..36941992b02 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -103,6 +103,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2419,6 +2422,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 4fd6574e129..7f7c7c16e23 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1048,6 +1048,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index d1768a89f6e..c283e442f6f 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -260,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -879,7 +878,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -945,7 +943,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1061,7 +1059,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1111,8 +1109,30 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 8a3f7d434cf..791d777fbc6 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -48,6 +48,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -204,12 +206,40 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.interrupts++;
+ pgstat_unlock_entry(entry_ref);
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -233,6 +263,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -861,6 +893,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -984,3 +1019,38 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->system_time += src->system_time;
+ dst->user_time += src->user_time;
+ dst->total_time += src->total_time;
+ dst->interrupts += src->interrupts;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->dead_tuples += src->dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index f7b50e0b5af..eba1783e51a 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -31,6 +31,42 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+#include "utils/pgstat_internal.h"
+
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -2063,3 +2099,123 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+#define EXTVACHEAPSTAT_COLUMNS 27
+
+static void
+tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
+ PgStat_StatTabEntry *tabentry)
+{
+ Datum values[EXTVACHEAPSTAT_COLUMNS];
+ bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_fetched -
+ tabentry->vacuum_ext.blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, tabentry->vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(tabentry->vacuum_ext.interrupts);
+
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+}
+
+/*
+ * Get the vacuum statistics for the heap tables or indexes.
+ */
+static void
+pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* Check if caller supports us returning a tuplestore */
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("set-valued function called in context that cannot accept a set")));
+ Assert(rsinfo->setDesc->natts == ncolumns);
+ Assert(rsinfo->setResult != NULL);
+
+ /* Load table statistics for specified database. */
+ if (OidIsValid(relid))
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ if (tabentry == NULL)
+ /* Table don't exists or isn't an heap relation. */
+ return;
+
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
+
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ tabentry = (PgStat_StatTabEntry *) entry->data;
+
+ if (tabentry != NULL)
+ tuplestore_put_for_relation(entry->key.objid, rsinfo, tabentry);
+ }
+ }
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 987ff98067b..ade2f154a71 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 77f54a79e6a..c861e5691cb 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12329,4 +12329,13 @@
proargtypes => 'int2',
prosrc => 'gist_stratnum_identity' },
+{ oid => '8001',
+ descr => 'pg_stat_vacuum_tables return stats values',
+ proname => 'pg_stat_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_tables' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index df53fa2d4f9..e764a8c5326 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,6 +169,52 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ int64 total_blks_read; /* number of pages that were missed in shared buffers during a vacuum of specific relation */
+ int64 total_blks_hit; /* number of pages that were found in shared buffers during a vacuum of specific relation */
+ int64 total_blks_dirtied; /* number of pages marked as 'Dirty' during a vacuum of specific relation. */
+ int64 total_blks_written; /* number of pages written during a vacuum of specific relation. */
+
+ int64 blks_fetched; /* number of a relation blocks, fetched during the vacuum. */
+ int64 blks_hit; /* number of a relation blocks, found in shared buffers during the vacuum. */
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double system_time; /* amount of time the CPU was busy executing vacuum code in kernel space, in msec */
+ double user_time; /* amount of time the CPU was busy executing vacuum code in user space, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 interrupts;
+
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -209,6 +255,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -267,7 +323,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB1
typedef struct PgStat_ArchiverStats
{
@@ -388,6 +444,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -461,6 +519,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -626,10 +689,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -677,6 +742,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -694,7 +770,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
/*
* Functions in pgstat_replslot.c
*/
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index e54eca5b489..e752c0ce015 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 61b2e1f96b2..2c0e55d63f3 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -573,7 +573,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
*/
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..7cdb79c0ec4
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 100| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 100| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..e93dd4f626c 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5facb2c862c
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,51 @@
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
+# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2b47013f113..700a4863964 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2237,6 +2237,40 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'r'::"char"));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..064064e94b2
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,209 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | t | t
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+UPDATE vestat SET x = x1001;
+ERROR: column "x1001" does not exist
+LINE 1: UPDATE vestat SET x = x1001;
+ ^
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | f | f
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ t | t | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
+ min
+-----
+ 112
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4f38104ba01..32c706d3363 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..bc8d051aefa
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,160 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+UPDATE vestat SET x = x1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v9-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (39.7K, 4-v9-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From dd2b403333a87e5754e0c2fb8d133b003fb56746 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 8 Oct 2024 19:11:18 +0300
Subject: [PATCH 2/4] Machinery for grabbing an extended vacuum statistics on
heap and index relations. Remember, statistic on heap and index relations a
bit different (see ExtVacReport to find out more information). The concept of
the ExtVacReport structure has been complicated to store statistic
information for two kinds of relations: for heap and index relations.
ExtVacReportType variable helps to determine what the kind is considering
now.
---
src/backend/access/heap/vacuumlazy.c | 99 +++++++++--
src/backend/catalog/system_views.sql | 40 +++++
src/backend/utils/activity/pgstat.c | 7 +-
src/backend/utils/activity/pgstat_relation.c | 41 +++--
src/backend/utils/adt/pgstatfuncs.c | 98 +++++++----
src/include/catalog/pg_proc.dat | 9 +
src/include/pgstat.h | 52 ++++--
.../vacuum-extending-in-repetable-read.out | 7 +-
src/test/regress/expected/rules.out | 26 +++
.../expected/vacuum_index_statistics.out | 164 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 9 +-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 130 ++++++++++++++
13 files changed, 605 insertions(+), 78 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d63303c7fb7..9c53d0b4c57 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -168,6 +168,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -246,6 +247,13 @@ typedef struct LVExtStatCounters
PgStat_Counter blocks_hit;
} LVExtStatCounters;
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -408,6 +416,46 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+static void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+static void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -711,14 +759,15 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(rel, &extVacCounters, &extVacReport);
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.pages_frozen = vacrel->set_frozen_pages;
- extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.type = PGSTAT_EXTVAC_HEAP;
+ extVacReport.heap.pages_scanned = vacrel->scanned_pages;
+ extVacReport.heap.pages_removed = vacrel->removed_pages;
+ extVacReport.heap.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.heap.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.heap.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.heap.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.heap.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.heap.index_vacuum_count = vacrel->num_index_scans;
/*
* Report results to the cumulative stats system, too.
@@ -2583,6 +2632,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2601,6 +2654,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2609,6 +2663,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, (void *) vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2633,6 +2694,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2652,12 +2717,20 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3274,7 +3347,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3291,7 +3364,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3307,16 +3380,22 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ec997531326..f5e4e1fbaa5 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1434,3 +1434,43 @@ WHERE
rel.oid = stats.relid AND
ns.oid = rel.relnamespace AND
rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_deleted,
+ stats.tuples_deleted,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stat_vacuum_indexes(rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace AND
+ rel.relkind = 'i';
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index c283e442f6f..843617eba25 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1122,7 +1122,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
PG_TRY();
{
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
- pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ if (kind == PGSTAT_KIND_RELATION)
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
}
PG_FINALLY();
{
@@ -1177,6 +1178,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 791d777fbc6..5c95363c04a 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -213,7 +213,7 @@ pgstat_drop_relation(Relation rel)
* ---------
*/
void
-pgstat_report_vacuum_error(Oid tableoid)
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -230,6 +230,7 @@ pgstat_report_vacuum_error(Oid tableoid)
tabentry = &shtabentry->stats;
tabentry->vacuum_ext.interrupts++;
+ tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
}
@@ -1042,15 +1043,31 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->pages_frozen += src->pages_frozen;
- dst->pages_all_visible += src->pages_all_visible;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->dead_tuples += src->dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index eba1783e51a..e698d637860 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2101,17 +2101,19 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
}
#define EXTVACHEAPSTAT_COLUMNS 27
+#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
static void
tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
PgStat_StatTabEntry *tabentry)
{
- Datum values[EXTVACHEAPSTAT_COLUMNS];
- bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ Datum values[EXTVACSTAT_COLUMNS];
+ bool nulls[EXTVACSTAT_COLUMNS];
char buf[256];
int i = 0;
- memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+ memset(nulls, 0, EXTVACSTAT_COLUMNS * sizeof(bool));
values[i++] = ObjectIdGetDatum(relid);
@@ -2124,16 +2126,25 @@ tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
tabentry->vacuum_ext.blks_hit);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
- values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
- values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+ if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_HEAP)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ }
+ else if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.pages_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.tuples_deleted);
+ }
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
@@ -2161,10 +2172,9 @@ tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
* Get the vacuum statistics for the heap tables or indexes.
*/
static void
-pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
{
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
- Oid relid = PG_GETARG_OID(0);
PgStat_StatTabEntry *tabentry;
InitMaterializedSRF(fcinfo, 0);
@@ -2177,34 +2187,39 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Assert(rsinfo->setDesc->natts == ncolumns);
Assert(rsinfo->setResult != NULL);
- /* Load table statistics for specified database. */
- if (OidIsValid(relid))
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
{
- tabentry = pgstat_fetch_stat_tabentry(relid);
- if (tabentry == NULL)
- /* Table don't exists or isn't an heap relation. */
- return;
+ Oid relid = PG_GETARG_OID(0);
- tuplestore_put_for_relation(relid, rsinfo, tabentry);
- }
- else
- {
- SnapshotIterator hashiter;
- PgStat_SnapshotEntry *entry;
+ /* Load table statistics for specified relation. */
+ if (OidIsValid(relid))
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ if (tabentry == NULL || tabentry->vacuum_ext.type != type)
+ /* Table don't exists or isn't an heap relation. */
+ return;
+
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
- pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
- /* Iterate the snapshot */
- InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
- while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
- {
- CHECK_FOR_INTERRUPTS();
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
- tabentry = (PgStat_StatTabEntry *) entry->data;
+ tabentry = (PgStat_StatTabEntry *) entry->data;
- if (tabentry != NULL)
- tuplestore_put_for_relation(entry->key.objid, rsinfo, tabentry);
+ if (tabentry != NULL && tabentry->vacuum_ext.type == type)
+ tuplestore_put_for_relation(entry->key.objoid, rsinfo, tabentry);
+ }
}
}
}
@@ -2217,5 +2232,16 @@ pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
{
pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+ PG_RETURN_VOID();
+}
+
+/*
+ * Get the vacuum statistics for the indexes.
+ */
+Datum
+pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
+
PG_RETURN_VOID();
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c861e5691cb..9723551a73a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12338,4 +12338,13 @@
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
prosrc => 'pg_stat_vacuum_tables' },
+{ oid => '8002',
+ descr => 'pg_stat_vacuum_indexes return stats values',
+ proname => 'pg_stat_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_indexes' }
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index e764a8c5326..b784bcc3efe 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,11 +169,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_HEAP = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -205,14 +213,38 @@ typedef struct ExtVacReport
/* Interruptions on any errors. */
int32 interrupts;
- int64 pages_scanned; /* number of pages we examined */
- int64 pages_removed; /* number of pages removed by vacuum */
- int64 pages_frozen; /* number of pages marked in VM as frozen */
- int64 pages_all_visible; /* number of pages marked in VM as all-visible */
- int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
- int64 index_vacuum_count; /* number of index vacuumings */
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } heap;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -694,7 +726,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 7cdb79c0ec4..93fe15c01f9 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -9,10 +9,9 @@ step s2_print_vacuum_stats_table:
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
-relname |tuples_deleted|dead_tuples|tuples_frozen
---------------------------+--------------+-----------+-------------
-test_vacuum_stat_isolation| 0| 0| 0
-(1 row)
+relname|tuples_deleted|dead_tuples|tuples_frozen
+-------+--------------+-----------+-------------
+(0 rows)
step s1_begin_repeatable_read:
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 700a4863964..e3290da748d 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2237,6 +2237,32 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'i'::"char"));
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..4f6e305710e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,164 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+---------+----------+---------------+----------------
+(0 rows)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_vacuum_indexes(0);
+ min
+------
+ 1232
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index 064064e94b2..86272217e5d 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -37,8 +37,7 @@ FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
---------+--------------+----------------+----------+---------------+---------------
- vestat | 0 | 0 | 455 | 0 | 0
-(1 row)
+(0 rows)
SELECT relpages AS rp
FROM pg_class c
@@ -201,9 +200,9 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
(1 row)
SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
- min
------
- 112
+ min
+------
+ 1213
(1 row)
DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 32c706d3363..34fd3e4e674 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..75e5974eb59
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,130 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+SELECT min(relid) FROM pg_stat_vacuum_indexes(0);
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] v9-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (18.5K, 5-v9-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 9b2b6ab2ae56d975f823db830c43a4520771662d Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 8 Oct 2024 19:13:05 +0300
Subject: [PATCH 3/4] Machinery for grabbing an extended vacuum statistics on
databases. It transmits vacuum statistical information about each table and
accumulates it for the database which the table belonged.
---
src/backend/catalog/system_views.sql | 28 +++++++
src/backend/utils/activity/pgstat.c | 2 +
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 16 ++++
src/backend/utils/adt/pgstatfuncs.c | 76 +++++++++++++++++-
src/include/catalog/pg_proc.dat | 11 ++-
src/include/pgstat.h | 3 +-
src/test/regress/expected/rules.out | 18 +++++
...ut => vacuum_tables_and_db_statistics.out} | 78 +++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 66 +++++++++++++++-
11 files changed, 295 insertions(+), 6 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (77%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (78%)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index f5e4e1fbaa5..b63c1804b41 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1474,3 +1474,31 @@ WHERE
rel.oid = stats.relid AND
ns.oid = rel.relnamespace AND
rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+
+ stats.interrupts
+FROM
+ pg_database db LEFT JOIN pg_stat_vacuum_database(db.oid) stats
+ON
+ db.oid = stats.dboid;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 843617eba25..21b29804620 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1124,6 +1124,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
if (kind == PGSTAT_KIND_RELATION)
pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ else if (kind == PGSTAT_KIND_DATABASE)
+ pgstat_build_snapshot(PGSTAT_KIND_DATABASE);
}
PG_FINALLY();
{
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 29bc0909748..a060d1a4042 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -430,6 +430,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 5c95363c04a..725e26423f2 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -219,6 +219,7 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
if (!pgstat_track_counts)
return;
@@ -232,6 +233,10 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
tabentry->vacuum_ext.interrupts++;
tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.interrupts++;
+ dbentry->vacuum_ext.type = m_type;
}
/*
@@ -245,6 +250,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -298,6 +304,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index e698d637860..4d1c099b37e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2102,8 +2102,49 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
#define EXTVACHEAPSTAT_COLUMNS 27
#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACDBSTAT_COLUMNS 15
#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
+static void
+tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
+ PgStatShared_Database *dbentry)
+{
+ Datum values[EXTVACDBSTAT_COLUMNS];
+ bool nulls[EXTVACDBSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACDBSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->stats.vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(dbentry->stats.vacuum_ext.interrupts);
+
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+}
+
static void
tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
PgStat_StatTabEntry *tabentry)
@@ -2218,10 +2259,30 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
tabentry = (PgStat_StatTabEntry *) entry->data;
if (tabentry != NULL && tabentry->vacuum_ext.type == type)
- tuplestore_put_for_relation(entry->key.objoid, rsinfo, tabentry);
+ tuplestore_put_for_relation(entry->key.objid, rsinfo, tabentry);
}
}
}
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ PgStatShared_Database *dbentry;
+ PgStat_EntryRef *entry_ref;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (OidIsValid(dbid))
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dbid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ if (dbentry == NULL)
+ /* Table doesn't exist or isn't a heap relation */
+ return;
+
+ tuplestore_put_for_database(dbid, rsinfo, dbentry);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
}
/*
@@ -2230,7 +2291,7 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
Datum
pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
{
- pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
PG_RETURN_VOID();
}
@@ -2243,5 +2304,16 @@ pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
{
pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
+ PG_RETURN_VOID();
+}
+
+/*
+ * Get the vacuum statistics for the database.
+ */
+Datum
+pg_stat_vacuum_database(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
+
PG_RETURN_VOID();
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9723551a73a..936713fe5c1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12346,5 +12346,14 @@
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
- prosrc => 'pg_stat_vacuum_indexes' }
+ prosrc => 'pg_stat_vacuum_indexes' },
+{ oid => '8003',
+ descr => 'pg_stat_vacuum_database return stats values',
+ proname => 'pg_stat_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index b784bcc3efe..c6d663c1c48 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -174,7 +174,8 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_HEAP = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index e3290da748d..8359cf3e984 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2237,6 +2237,24 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM (pg_database db
+ LEFT JOIN LATERAL pg_stat_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON ((db.oid = stats.dboid)));
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 77%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index 86272217e5d..94dd3214349 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,6 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -205,4 +208,79 @@ SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
1213
(1 row)
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 34fd3e4e674..e999232d429 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 78%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index bc8d051aefa..af1281b3b63 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,6 +7,10 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+
-- conditio sine qua non
SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
@@ -157,4 +161,64 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+DROP TABLE vestat CASCADE;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
--
2.34.1
[text/x-patch] v9-0004-Add-documentation-about-the-system-views-that-are-us.patch (24.2K, 6-v9-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 55a6e8c7dcd9cb3ec9c4e87a13ee9c5bd57183bf Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:47:55 +0300
Subject: [PATCH 4/4] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 747 +++++++++++++++++++++++++++++++++
1 file changed, 747 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 61d28e701f2..93fe9fe36c7 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5064,4 +5064,751 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stats-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this index
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-frozen in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_all_visible</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-visible in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this table
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
+
</chapter>
--
2.34.1
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-10-16 10:31 Ilia Evdokimov <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 2 replies; 77+ messages in thread
From: Ilia Evdokimov @ 2024-10-16 10:31 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; pgsql-hackers; +Cc: jian he <[email protected]>; Alexander Korotkov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; [email protected]
On 08.10.2024 19:18, Alena Rybakina wrote:
> Made a rebase on a fresh master branch.
> --
> Regards,
> Alena Rybakina
> Postgres Professional
Thank you for rebasing.
I have noticed that when I create a table or an index on this table,
there is no information about the table or index in
pg_stat_vacuum_tables and pg_stat_vacuum_indexes until we perform a VACUUM.
Example:
CREATE TABLE t (i INT, j INT);
INSERT INTO t SELECT i/10, i/100 FROM GENERATE_SERIES(1,1000000) i;
SELECT * FROM pg_stat_vacuum_tables WHERE relname = 't';
....
(0 rows)
CREATE INDEX ON t (i);
SELECT * FROM pg_stat_vacuum_indexes WHERE relname = 't_i_idx';
...
(0 rows)
I can see the entries after running VACUUM or executing autovacuum. or
when autovacuum is executed. I would suggest adding a line about the
relation even if it has not yet been processed by vacuum. Interestingly,
this issue does not occur with pg_stat_vacuum_database:
CREATE DATABASE example_db;
SELECT * FROM pg_stat_vacuum_database WHERE dbname = 'example_db';
dboid | dbname | ...
... | example_db | ...
(1 row)
BTW, I recommend renaming the view pg_stat_vacuum_database to
pg_stat_vacuum_database_S_ for consistency with pg_stat_vacuum_tables
and pg_stat_vacuum_indexes
--
Regards,
Ilia Evdokimov,
Tantor Labs LLC.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-10-16 11:01 Alena Rybakina <[email protected]>
parent: Ilia Evdokimov <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2024-10-16 11:01 UTC (permalink / raw)
To: Ilia Evdokimov <[email protected]>; pgsql-hackers; +Cc: jian he <[email protected]>; Alexander Korotkov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; [email protected]
Hi!
On 16.10.2024 13:31, Ilia Evdokimov wrote:
>
>
> On 08.10.2024 19:18, Alena Rybakina wrote:
>> Made a rebase on a fresh master branch.
>> --
>> Regards,
>> Alena Rybakina
>> Postgres Professional
>
> Thank you for rebasing.
>
> I have noticed that when I create a table or an index on this table,
> there is no information about the table or index in
> pg_stat_vacuum_tables and pg_stat_vacuum_indexes until we perform a
> VACUUM.
>
> Example:
>
> CREATE TABLE t (i INT, j INT);
> INSERT INTO t SELECT i/10, i/100 FROM GENERATE_SERIES(1,1000000) i;
> SELECT * FROM pg_stat_vacuum_tables WHERE relname = 't';
> ....
> (0 rows)
> CREATE INDEX ON t (i);
> SELECT * FROM pg_stat_vacuum_indexes WHERE relname = 't_i_idx';
> ...
> (0 rows)
>
> I can see the entries after running VACUUM or executing autovacuum. or
> when autovacuum is executed. I would suggest adding a line about the
> relation even if it has not yet been processed by
> vacuum. Interestingly, this issue does not occur with
> pg_stat_vacuum_database:
>
> CREATE DATABASE example_db;
> SELECT * FROM pg_stat_vacuum_database WHERE dbname = 'example_db';
> dboid | dbname | ...
> ... | example_db | ...
> (1 row)
>
> BTW, I recommend renaming the view pg_stat_vacuum_database to
> pg_stat_vacuum_database_S_ for consistency with pg_stat_vacuum_tables
> and pg_stat_vacuum_indexes
>
Thanks for the review. I'm investigating this. I agree with the
renaming, I will do it in the next version of the patch.
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-10-16 11:17 Andrei Zubkov <[email protected]>
parent: Ilia Evdokimov <[email protected]>
1 sibling, 0 replies; 77+ messages in thread
From: Andrei Zubkov @ 2024-10-16 11:17 UTC (permalink / raw)
To: Ilia Evdokimov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; +Cc: jian he <[email protected]>; Alexander Korotkov <[email protected]>; Alena Rybakina <[email protected]>; [email protected]
Hi Ilia,
On Wed, 2024-10-16 at 13:31 +0300, Ilia Evdokimov wrote:
> BTW, I recommend renaming the view pg_stat_vacuum_database to
> pg_stat_vacuum_databaseS for consistency with pg_stat_vacuum_tables
> and pg_stat_vacuum_indexes
Such renaming doesn't seems correct to me because
pg_stat_vacuum_database is consistent with pg_stat_database view, while
pg_stat_vacuum_tables is consistent with pg_stat_all_tables.
This inconsistency is in Postgres views, so it should be changed
synchronously.
--
regards, Andrei Zubkov
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-10-22 19:30 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2024-10-22 19:30 UTC (permalink / raw)
To: Ilia Evdokimov <[email protected]>; pgsql-hackers; +Cc: jian he <[email protected]>; Alexander Korotkov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; [email protected]
Hi!
On 16.10.2024 14:01, Alena Rybakina wrote:
>>
>> Thank you for rebasing.
>>
>> I have noticed that when I create a table or an index on this table,
>> there is no information about the table or index in
>> pg_stat_vacuum_tables and pg_stat_vacuum_indexes until we perform a
>> VACUUM.
>>
>> Example:
>>
>> CREATE TABLE t (i INT, j INT);
>> INSERT INTO t SELECT i/10, i/100 FROM GENERATE_SERIES(1,1000000) i;
>> SELECT * FROM pg_stat_vacuum_tables WHERE relname = 't';
>> ....
>> (0 rows)
>> CREATE INDEX ON t (i);
>> SELECT * FROM pg_stat_vacuum_indexes WHERE relname = 't_i_idx';
>> ...
>> (0 rows)
>>
>> I can see the entries after running VACUUM or executing
>> autovacuum. or when autovacuum is executed. I would suggest adding a
>> line about the relation even if it has not yet been processed by
>> vacuum. Interestingly, this issue does not occur with
>> pg_stat_vacuum_database:
>>
>> CREATE DATABASE example_db;
>> SELECT * FROM pg_stat_vacuum_database WHERE dbname = 'example_db';
>> dboid | dbname | ...
>> ... | example_db | ...
>> (1 row)
>>
>> BTW, I recommend renaming the view pg_stat_vacuum_database to
>> pg_stat_vacuum_database_S_ for consistency with
>> pg_stat_vacuum_tables and pg_stat_vacuum_indexes
>>
> Thanks for the review. I'm investigating this. I agree with the
> renaming, I will do it in the next version of the patch.
>
I fixed it. I added the left outer join to the vacuum views and for
converting the coalesce function from NULL to null values.
I also fixed the code in getting database statistics - we can get it
through the existing pgstat_fetch_stat_dbentry function and fixed couple
of comments.
I attached a diff file, as well as new versions of patches.
--
Regards,
Alena Rybakina
Postgres Professional
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index b63c1804b41..b68e0f00abd 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1392,48 +1392,42 @@ SELECT
ns.nspname AS "schema",
rel.relname AS relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
-
- stats.rel_blks_read,
- stats.rel_blks_hit,
-
- stats.pages_scanned,
- stats.pages_removed,
- stats.pages_frozen,
- stats.pages_all_visible,
- stats.tuples_deleted,
- stats.tuples_frozen,
- stats.dead_tuples,
-
- stats.index_vacuum_count,
- stats.rev_all_frozen_pages,
- stats.rev_all_visible_pages,
-
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
-
- stats.blk_read_time,
- stats.blk_write_time,
-
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
- stats.interrupts
-FROM
- pg_database db,
- pg_class rel,
- pg_namespace ns,
- pg_stat_vacuum_tables(rel.oid) stats
-WHERE
- db.datname = current_database() AND
- rel.oid = stats.relid AND
- ns.oid = rel.relnamespace AND
- rel.relkind = 'r';
+ COALESCE(stats.total_blks_read, 0) AS total_blks_read,
+ COALESCE(stats.total_blks_hit, 0) AS total_blks_hit,
+ COALESCE(stats.total_blks_dirtied, 0) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, 0) AS total_blks_written,
+
+ COALESCE(stats.rel_blks_read, 0) AS rel_blks_read,
+ COALESCE(stats.rel_blks_hit, 0) AS rel_blks_hit,
+
+ COALESCE(stats.pages_scanned, 0) AS pages_scanned,
+ COALESCE(stats.pages_removed, 0) AS pages_removed,
+ COALESCE(stats.pages_frozen, 0) AS pages_frozen,
+ COALESCE(stats.pages_all_visible, 0) AS pages_all_visible,
+ COALESCE(stats.tuples_deleted, 0) AS tuples_deleted,
+ COALESCE(stats.tuples_frozen, 0) AS tuples_frozen,
+ COALESCE(stats.dead_tuples, 0) AS dead_tuples,
+
+ COALESCE(stats.index_vacuum_count, 0) AS index_vacuum_count,
+ COALESCE(stats.rev_all_frozen_pages, 0) AS rev_all_frozen_pages,
+ COALESCE(stats.rev_all_visible_pages, 0) AS rev_all_visible_pages,
+
+ COALESCE(stats.wal_records, 0) AS wal_records,
+ COALESCE(stats.wal_fpi, 0) AS wal_fpi,
+ COALESCE(stats.wal_bytes, 0) AS wal_bytes,
+
+ COALESCE(stats.blk_read_time, 0) AS blk_read_time,
+ COALESCE(stats.blk_write_time, 0) AS blk_write_time,
+
+ COALESCE(stats.delay_time, 0) AS delay_time,
+ COALESCE(stats.system_time, 0) AS system_time,
+ COALESCE(stats.user_time, 0) AS user_time,
+ COALESCE(stats.total_time, 0) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace
+ LEFT JOIN pg_stat_vacuum_tables(rel.oid) stats ON true
+WHERE rel.relkind = 'r';
CREATE VIEW pg_stat_vacuum_indexes AS
SELECT
@@ -1441,64 +1435,57 @@ SELECT
ns.nspname AS "schema",
rel.relname AS relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
+ COALESCE(total_blks_read, 0) AS total_blks_read,
+ COALESCE(total_blks_hit, 0) AS total_blks_hit,
+ COALESCE(total_blks_dirtied, 0) AS total_blks_dirtied,
+ COALESCE(total_blks_written, 0) AS total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
+ COALESCE(rel_blks_read, 0) AS rel_blks_read,
+ COALESCE(rel_blks_hit, 0) AS rel_blks_hit,
- stats.pages_deleted,
- stats.tuples_deleted,
+ COALESCE(pages_deleted, 0) AS pages_deleted,
+ COALESCE(tuples_deleted, 0) AS tuples_deleted,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
+ COALESCE(wal_records, 0) AS wal_records,
+ COALESCE(wal_fpi, 0) AS wal_fpi,
+ COALESCE(wal_bytes, 0) AS wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
+ COALESCE(blk_read_time, 0) AS blk_read_time,
+ COALESCE(blk_write_time, 0) AS blk_write_time,
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
- stats.interrupts
+ COALESCE(delay_time, 0) AS delay_time,
+ COALESCE(system_time, 0) AS system_time,
+ COALESCE(user_time, 0) AS user_time,
+ COALESCE(total_time, 0) AS total_time,
+ COALESCE(interrupts, 0) AS interrupts
FROM
- pg_database db,
- pg_class rel,
- pg_namespace ns,
- pg_stat_vacuum_indexes(rel.oid) stats
-WHERE
- db.datname = current_database() AND
- rel.oid = stats.relid AND
- ns.oid = rel.relnamespace AND
- rel.relkind = 'i';
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace
+ LEFT JOIN pg_stat_vacuum_indexes(rel.oid) stats ON true
+WHERE rel.relkind = 'i';
CREATE VIEW pg_stat_vacuum_database AS
SELECT
db.oid as dboid,
db.datname AS dbname,
- stats.db_blks_read,
- stats.db_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
-
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
+ COALESCE(stats.db_blks_read, 0) AS db_blks_read,
+ COALESCE(stats.db_blks_hit, 0) AS db_blks_hit,
+ COALESCE(stats.total_blks_dirtied, 0) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, 0) AS total_blks_written,
- stats.blk_read_time,
- stats.blk_write_time,
+ COALESCE(stats.wal_records, 0) AS wal_records,
+ COALESCE(stats.wal_fpi, 0) AS wal_fpi,
+ COALESCE(stats.wal_bytes, 0) AS wal_bytes,
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
+ COALESCE(stats.blk_read_time, 0) AS blk_read_time,
+ COALESCE(stats.blk_write_time, 0) AS blk_write_time,
- stats.interrupts
+ COALESCE(stats.delay_time, 0) AS delay_time,
+ COALESCE(stats.system_time, 0) AS system_time,
+ COALESCE(stats.user_time, 0) AS user_time,
+ COALESCE(stats.total_time, 0) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
FROM
- pg_database db LEFT JOIN pg_stat_vacuum_database(db.oid) stats
-ON
- db.oid = stats.dboid;
\ No newline at end of file
+ pg_database db
+ LEFT JOIN pg_stat_vacuum_database(db.oid) stats ON true;
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 4d1c099b37e..cac34fbe64f 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2107,7 +2107,7 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
static void
tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
- PgStatShared_Database *dbentry)
+ PgStat_StatDBEntry *dbentry)
{
Datum values[EXTVACDBSTAT_COLUMNS];
bool nulls[EXTVACDBSTAT_COLUMNS];
@@ -2118,28 +2118,28 @@ tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
values[i++] = ObjectIdGetDatum(dbid);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_read);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_hit);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_dirtied);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_written);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_written);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_records);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_fpi);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->stats.vacuum_ext.wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->vacuum_ext.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_read_time);
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_write_time);
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.delay_time);
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.system_time);
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.user_time);
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.total_time);
- values[i++] = Int32GetDatum(dbentry->stats.vacuum_ext.interrupts);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(dbentry->vacuum_ext.interrupts);
Assert(i == rsinfo->setDesc->natts);
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
@@ -2236,8 +2236,9 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
if (OidIsValid(relid))
{
tabentry = pgstat_fetch_stat_tabentry(relid);
- if (tabentry == NULL || tabentry->vacuum_ext.type != type)
- /* Table don't exists or isn't an heap relation. */
+
+ if ((tabentry == NULL || tabentry->vacuum_ext.type != type))
+ /* Table don't exists or isn't a heap or index relation. */
return;
tuplestore_put_for_relation(relid, rsinfo, tabentry);
@@ -2245,7 +2246,7 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
else
{
SnapshotIterator hashiter;
- PgStat_SnapshotEntry *entry;
+ PgStat_SnapshotEntry *entry;
pgstat_update_snapshot(PGSTAT_KIND_RELATION);
@@ -2265,22 +2266,18 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
}
else if (type == PGSTAT_EXTVAC_DB)
{
- PgStatShared_Database *dbentry;
- PgStat_EntryRef *entry_ref;
- Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ Oid dbid = PG_GETARG_OID(0);
if (OidIsValid(dbid))
{
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dbid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
if (dbentry == NULL)
- /* Table doesn't exist or isn't a heap relation */
+ /* Database doesn't exist */
return;
tuplestore_put_for_database(dbid, rsinfo, dbentry);
- pgstat_unlock_entry(entry_ref);
}
}
}
@@ -2316,4 +2313,4 @@ pg_stat_vacuum_database(PG_FUNCTION_ARGS)
pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
PG_RETURN_VOID();
-}
\ No newline at end of file
+ }
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 8359cf3e984..f8112d54f52 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2239,82 +2239,80 @@ pg_stat_user_tables| SELECT relid,
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_vacuum_database| SELECT db.oid AS dboid,
db.datname AS dbname,
- stats.db_blks_read,
- stats.db_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
- stats.interrupts
+ COALESCE(stats.db_blks_read, (0)::bigint) AS db_blks_read,
+ COALESCE(stats.db_blks_hit, (0)::bigint) AS db_blks_hit,
+ COALESCE(stats.total_blks_dirtied, (0)::bigint) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, (0)::bigint) AS total_blks_written,
+ COALESCE(stats.wal_records, (0)::bigint) AS wal_records,
+ COALESCE(stats.wal_fpi, (0)::bigint) AS wal_fpi,
+ COALESCE(stats.wal_bytes, (0)::numeric) AS wal_bytes,
+ COALESCE(stats.blk_read_time, (0)::double precision) AS blk_read_time,
+ COALESCE(stats.blk_write_time, (0)::double precision) AS blk_write_time,
+ COALESCE(stats.delay_time, (0)::double precision) AS delay_time,
+ COALESCE(stats.system_time, (0)::double precision) AS system_time,
+ COALESCE(stats.user_time, (0)::double precision) AS user_time,
+ COALESCE(stats.total_time, (0)::double precision) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
FROM (pg_database db
- LEFT JOIN LATERAL pg_stat_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON ((db.oid = stats.dboid)));
+ LEFT JOIN LATERAL pg_stat_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON (true));
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_deleted,
- stats.tuples_deleted,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
- stats.interrupts
- FROM pg_database db,
- pg_class rel,
- pg_namespace ns,
- LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
- WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'i'::"char"));
+ COALESCE(stats.total_blks_read, (0)::bigint) AS total_blks_read,
+ COALESCE(stats.total_blks_hit, (0)::bigint) AS total_blks_hit,
+ COALESCE(stats.total_blks_dirtied, (0)::bigint) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, (0)::bigint) AS total_blks_written,
+ COALESCE(stats.rel_blks_read, (0)::bigint) AS rel_blks_read,
+ COALESCE(stats.rel_blks_hit, (0)::bigint) AS rel_blks_hit,
+ COALESCE(stats.pages_deleted, (0)::bigint) AS pages_deleted,
+ COALESCE(stats.tuples_deleted, (0)::bigint) AS tuples_deleted,
+ COALESCE(stats.wal_records, (0)::bigint) AS wal_records,
+ COALESCE(stats.wal_fpi, (0)::bigint) AS wal_fpi,
+ COALESCE(stats.wal_bytes, (0)::numeric) AS wal_bytes,
+ COALESCE(stats.blk_read_time, (0)::double precision) AS blk_read_time,
+ COALESCE(stats.blk_write_time, (0)::double precision) AS blk_write_time,
+ COALESCE(stats.delay_time, (0)::double precision) AS delay_time,
+ COALESCE(stats.system_time, (0)::double precision) AS system_time,
+ COALESCE(stats.user_time, (0)::double precision) AS user_time,
+ COALESCE(stats.total_time, (0)::double precision) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+ FROM ((pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace)))
+ LEFT JOIN LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON (true))
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_scanned,
- stats.pages_removed,
- stats.pages_frozen,
- stats.pages_all_visible,
- stats.tuples_deleted,
- stats.tuples_frozen,
- stats.dead_tuples,
- stats.index_vacuum_count,
- stats.rev_all_frozen_pages,
- stats.rev_all_visible_pages,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
- stats.interrupts
- FROM pg_database db,
- pg_class rel,
- pg_namespace ns,
- LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
- WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'r'::"char"));
+ COALESCE(stats.total_blks_read, (0)::bigint) AS total_blks_read,
+ COALESCE(stats.total_blks_hit, (0)::bigint) AS total_blks_hit,
+ COALESCE(stats.total_blks_dirtied, (0)::bigint) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, (0)::bigint) AS total_blks_written,
+ COALESCE(stats.rel_blks_read, (0)::bigint) AS rel_blks_read,
+ COALESCE(stats.rel_blks_hit, (0)::bigint) AS rel_blks_hit,
+ COALESCE(stats.pages_scanned, (0)::bigint) AS pages_scanned,
+ COALESCE(stats.pages_removed, (0)::bigint) AS pages_removed,
+ COALESCE(stats.pages_frozen, (0)::bigint) AS pages_frozen,
+ COALESCE(stats.pages_all_visible, (0)::bigint) AS pages_all_visible,
+ COALESCE(stats.tuples_deleted, (0)::bigint) AS tuples_deleted,
+ COALESCE(stats.tuples_frozen, (0)::bigint) AS tuples_frozen,
+ COALESCE(stats.dead_tuples, (0)::bigint) AS dead_tuples,
+ COALESCE(stats.index_vacuum_count, (0)::bigint) AS index_vacuum_count,
+ COALESCE(stats.rev_all_frozen_pages, (0)::bigint) AS rev_all_frozen_pages,
+ COALESCE(stats.rev_all_visible_pages, (0)::bigint) AS rev_all_visible_pages,
+ COALESCE(stats.wal_records, (0)::bigint) AS wal_records,
+ COALESCE(stats.wal_fpi, (0)::bigint) AS wal_fpi,
+ COALESCE(stats.wal_bytes, (0)::numeric) AS wal_bytes,
+ COALESCE(stats.blk_read_time, (0)::double precision) AS blk_read_time,
+ COALESCE(stats.blk_write_time, (0)::double precision) AS blk_write_time,
+ COALESCE(stats.delay_time, (0)::double precision) AS delay_time,
+ COALESCE(stats.system_time, (0)::double precision) AS system_time,
+ COALESCE(stats.user_time, (0)::double precision) AS user_time,
+ COALESCE(stats.total_time, (0)::double precision) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+ FROM ((pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace)))
+ LEFT JOIN LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON (true))
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index 4f6e305710e..166de176e29 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -35,9 +35,10 @@ DELETE FROM vestat WHERE x % 2 = 0;
SELECT vt.relname,relpages,pages_deleted,tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
----------+----------+---------------+----------------
-(0 rows)
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
SELECT relpages AS irp
FROM pg_class c
diff --git a/src/test/regress/expected/vacuum_tables_and_db_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
index 94dd3214349..7a0b3ba96e1 100644
--- a/src/test/regress/expected/vacuum_tables_and_db_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -26,8 +26,6 @@ SELECT pg_stat_force_next_flush();
(1 row)
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -40,7 +38,8 @@ FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
---------+--------------+----------------+----------+---------------+---------------
-(0 rows)
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
SELECT relpages AS rp
FROM pg_class c
@@ -179,10 +178,7 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
-UPDATE vestat SET x = x1001;
-ERROR: column "x1001" does not exist
-LINE 1: UPDATE vestat SET x = x1001;
- ^
+UPDATE vestat SET x = x+1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
@@ -259,8 +255,6 @@ WHERE dbname = 'regression_statistic_vacuum_db';
(1 row)
\c regression_statistic_vacuum_db
-RESET vacuum_freeze_min_age;
-RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
\c regression_statistic_vacuum_db1;
SELECT count(*)
diff --git a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index af1281b3b63..a3ddc9419de 100644
--- a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -21,8 +21,7 @@ SET track_functions TO 'all';
SELECT pg_stat_force_next_flush();
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
+
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -145,7 +144,7 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
-UPDATE vestat SET x = x1001;
+UPDATE vestat SET x = x+1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
@@ -204,8 +203,6 @@ WHERE dbname = 'regression_statistic_vacuum_db';
\c regression_statistic_vacuum_db
-RESET vacuum_freeze_min_age;
-RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
\c regression_statistic_vacuum_db1;
Attachments:
[text/x-patch] v10-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (65.3K, 3-v10-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 748e194816f6292a6ff5fcb7d8e739a505a03719 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 8 Oct 2024 18:32:54 +0300
Subject: [PATCH 1/3] Machinery for grabbing an extended vacuum statistics on
heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 159 ++++++++++++-
src/backend/access/heap/visibilitymap.c | 13 ++
src/backend/catalog/system_views.sql | 50 +++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 32 ++-
src/backend/utils/activity/pgstat_relation.c | 72 +++++-
src/backend/utils/adt/pgstatfuncs.c | 156 +++++++++++++
src/backend/utils/error/elog.c | 13 ++
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 81 ++++++-
src/include/utils/elog.h | 1 +
src/include/utils/pgstat_internal.h | 2 +-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 51 +++++
src/test/regress/expected/rules.out | 33 +++
.../expected/vacuum_tables_statistics.out | 209 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 160 ++++++++++++++
21 files changed, 1093 insertions(+), 13 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d82aa3d4896..d63303c7fb7 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -167,6 +167,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -194,6 +195,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -226,6 +229,22 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Cut-off values of parameters which changes implicitly during a vacuum
+ * process.
+ * Vacuum can't control their values, so we should store them before and after
+ * the processing.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz time;
+ PGRUsage ru;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -279,6 +298,115 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+ PGRUsage ru0;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ pg_rusage_init(&ru0);
+ starttime = GetCurrentTimestamp();
+
+ counters->ru = ru0;
+ counters->time = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+ PGRUsage ru1;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->time, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ /*
+ * Get difference of a system time and user time values in milliseconds.
+ * Use floating point representation to show tails of time diffs.
+ */
+ pg_rusage_init(&ru1);
+ report->system_time =
+ (ru1.ru.ru_stime.tv_sec - counters->ru.ru.ru_stime.tv_sec) * 1000. +
+ (ru1.ru.ru_stime.tv_usec - counters->ru.ru.ru_stime.tv_usec) * 0.001;
+ report->user_time =
+ (ru1.ru.ru_utime.tv_sec - counters->ru.ru.ru_utime.tv_sec) * 1000. +
+ (ru1.ru.ru_utime.tv_usec - counters->ru.ru.ru_utime.tv_usec) * 0.001;
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -311,6 +439,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -329,7 +459,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -346,6 +476,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -413,6 +544,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -574,6 +707,19 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -588,7 +734,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1380,6 +1527,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2277,11 +2426,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -3122,6 +3273,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3137,6 +3290,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33c..d72cade60a4 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,18 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Initially, it didn't matter what type of flags (all-visible or frozen) we received,
+ * we just performed a reverse concatenation operation. But this information is very important
+ * for vacuum statistics. We need to find out this usingthe bit concatenation operation
+ * with the VISIBILITYMAP_ALL_VISIBLE and VISIBILITYMAP_ALL_FROZEN masks,
+ * and where the desired one matches, we increment the value there.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 3456b821bc5..4fa9886f409 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1379,3 +1379,53 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ COALESCE(stats.total_blks_read, 0) AS total_blks_read,
+ COALESCE(stats.total_blks_hit, 0) AS total_blks_hit,
+ COALESCE(stats.total_blks_dirtied, 0) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, 0) AS total_blks_written,
+
+ COALESCE(stats.rel_blks_read, 0) AS rel_blks_read,
+ COALESCE(stats.rel_blks_hit, 0) AS rel_blks_hit,
+
+ COALESCE(stats.pages_scanned, 0) AS pages_scanned,
+ COALESCE(stats.pages_removed, 0) AS pages_removed,
+ COALESCE(stats.pages_frozen, 0) AS pages_frozen,
+ COALESCE(stats.pages_all_visible, 0) AS pages_all_visible,
+ COALESCE(stats.tuples_deleted, 0) AS tuples_deleted,
+ COALESCE(stats.tuples_frozen, 0) AS tuples_frozen,
+ COALESCE(stats.dead_tuples, 0) AS dead_tuples,
+
+ COALESCE(stats.index_vacuum_count, 0) AS index_vacuum_count,
+ COALESCE(stats.rev_all_frozen_pages, 0) AS rev_all_frozen_pages,
+ COALESCE(stats.rev_all_visible_pages, 0) AS rev_all_visible_pages,
+
+ COALESCE(stats.wal_records, 0) AS wal_records,
+ COALESCE(stats.wal_fpi, 0) AS wal_fpi,
+ COALESCE(stats.wal_bytes, 0) AS wal_bytes,
+
+ COALESCE(stats.blk_read_time, 0) AS blk_read_time,
+ COALESCE(stats.blk_write_time, 0) AS blk_write_time,
+
+ COALESCE(stats.delay_time, 0) AS delay_time,
+ COALESCE(stats.system_time, 0) AS system_time,
+ COALESCE(stats.user_time, 0) AS user_time,
+ COALESCE(stats.total_time, 0) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace
+ LEFT JOIN pg_stat_vacuum_tables(rel.oid) stats ON true
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ac8f5d9c259..36941992b02 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -103,6 +103,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2419,6 +2422,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 4fd6574e129..7f7c7c16e23 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1048,6 +1048,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index d1768a89f6e..c283e442f6f 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -260,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -879,7 +878,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -945,7 +943,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1061,7 +1059,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1111,8 +1109,30 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 8a3f7d434cf..791d777fbc6 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -48,6 +48,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -204,12 +206,40 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.interrupts++;
+ pgstat_unlock_entry(entry_ref);
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -233,6 +263,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -861,6 +893,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -984,3 +1019,38 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->system_time += src->system_time;
+ dst->user_time += src->user_time;
+ dst->total_time += src->total_time;
+ dst->interrupts += src->interrupts;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->dead_tuples += src->dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index f7b50e0b5af..eba1783e51a 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -31,6 +31,42 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+#include "utils/pgstat_internal.h"
+
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -2063,3 +2099,123 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+#define EXTVACHEAPSTAT_COLUMNS 27
+
+static void
+tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
+ PgStat_StatTabEntry *tabentry)
+{
+ Datum values[EXTVACHEAPSTAT_COLUMNS];
+ bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_fetched -
+ tabentry->vacuum_ext.blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, tabentry->vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(tabentry->vacuum_ext.interrupts);
+
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+}
+
+/*
+ * Get the vacuum statistics for the heap tables or indexes.
+ */
+static void
+pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* Check if caller supports us returning a tuplestore */
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("set-valued function called in context that cannot accept a set")));
+ Assert(rsinfo->setDesc->natts == ncolumns);
+ Assert(rsinfo->setResult != NULL);
+
+ /* Load table statistics for specified database. */
+ if (OidIsValid(relid))
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ if (tabentry == NULL)
+ /* Table don't exists or isn't an heap relation. */
+ return;
+
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
+
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ tabentry = (PgStat_StatTabEntry *) entry->data;
+
+ if (tabentry != NULL)
+ tuplestore_put_for_relation(entry->key.objid, rsinfo, tabentry);
+ }
+ }
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 8acca3e0a0b..fe554547f5b 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7c0b74fe055..776a7344285 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12363,4 +12363,13 @@
proargtypes => 'int2',
prosrc => 'gist_stratnum_identity' },
+{ oid => '8001',
+ descr => 'pg_stat_vacuum_tables return stats values',
+ proname => 'pg_stat_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_tables' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index df53fa2d4f9..e764a8c5326 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,6 +169,52 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ int64 total_blks_read; /* number of pages that were missed in shared buffers during a vacuum of specific relation */
+ int64 total_blks_hit; /* number of pages that were found in shared buffers during a vacuum of specific relation */
+ int64 total_blks_dirtied; /* number of pages marked as 'Dirty' during a vacuum of specific relation. */
+ int64 total_blks_written; /* number of pages written during a vacuum of specific relation. */
+
+ int64 blks_fetched; /* number of a relation blocks, fetched during the vacuum. */
+ int64 blks_hit; /* number of a relation blocks, found in shared buffers during the vacuum. */
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double system_time; /* amount of time the CPU was busy executing vacuum code in kernel space, in msec */
+ double user_time; /* amount of time the CPU was busy executing vacuum code in user space, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 interrupts;
+
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -209,6 +255,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -267,7 +323,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB1
typedef struct PgStat_ArchiverStats
{
@@ -388,6 +444,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -461,6 +519,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -626,10 +689,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -677,6 +742,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -694,7 +770,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
/*
* Functions in pgstat_replslot.c
*/
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index e54eca5b489..e752c0ce015 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 61b2e1f96b2..2c0e55d63f3 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -573,7 +573,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
*/
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..7cdb79c0ec4
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 100| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 100| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..e93dd4f626c 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5facb2c862c
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,51 @@
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
+# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2b47013f113..61df9cfc64c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2237,6 +2237,39 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ COALESCE(stats.total_blks_read, (0)::bigint) AS total_blks_read,
+ COALESCE(stats.total_blks_hit, (0)::bigint) AS total_blks_hit,
+ COALESCE(stats.total_blks_dirtied, (0)::bigint) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, (0)::bigint) AS total_blks_written,
+ COALESCE(stats.rel_blks_read, (0)::bigint) AS rel_blks_read,
+ COALESCE(stats.rel_blks_hit, (0)::bigint) AS rel_blks_hit,
+ COALESCE(stats.pages_scanned, (0)::bigint) AS pages_scanned,
+ COALESCE(stats.pages_removed, (0)::bigint) AS pages_removed,
+ COALESCE(stats.pages_frozen, (0)::bigint) AS pages_frozen,
+ COALESCE(stats.pages_all_visible, (0)::bigint) AS pages_all_visible,
+ COALESCE(stats.tuples_deleted, (0)::bigint) AS tuples_deleted,
+ COALESCE(stats.tuples_frozen, (0)::bigint) AS tuples_frozen,
+ COALESCE(stats.dead_tuples, (0)::bigint) AS dead_tuples,
+ COALESCE(stats.index_vacuum_count, (0)::bigint) AS index_vacuum_count,
+ COALESCE(stats.rev_all_frozen_pages, (0)::bigint) AS rev_all_frozen_pages,
+ COALESCE(stats.rev_all_visible_pages, (0)::bigint) AS rev_all_visible_pages,
+ COALESCE(stats.wal_records, (0)::bigint) AS wal_records,
+ COALESCE(stats.wal_fpi, (0)::bigint) AS wal_fpi,
+ COALESCE(stats.wal_bytes, (0)::numeric) AS wal_bytes,
+ COALESCE(stats.blk_read_time, (0)::double precision) AS blk_read_time,
+ COALESCE(stats.blk_write_time, (0)::double precision) AS blk_write_time,
+ COALESCE(stats.delay_time, (0)::double precision) AS delay_time,
+ COALESCE(stats.system_time, (0)::double precision) AS system_time,
+ COALESCE(stats.user_time, (0)::double precision) AS user_time,
+ COALESCE(stats.total_time, (0)::double precision) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+ FROM ((pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace)))
+ LEFT JOIN LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON (true))
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..064064e94b2
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,209 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | t | t
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+UPDATE vestat SET x = x1001;
+ERROR: column "x1001" does not exist
+LINE 1: UPDATE vestat SET x = x1001;
+ ^
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | f | f
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ t | t | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
+ min
+-----
+ 112
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 81e4222d26a..977a0472027 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..bc8d051aefa
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,160 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+UPDATE vestat SET x = x1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v10-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (41.6K, 4-v10-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 92c555abed6e247bae171990a273147b7b3302cd Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 8 Oct 2024 19:11:18 +0300
Subject: [PATCH 2/3] Machinery for grabbing an extended vacuum statistics on
heap and index relations. Remember, statistic on heap and index relations a
bit different (see ExtVacReport to find out more information). The concept of
the ExtVacReport structure has been complicated to store statistic
information for two kinds of relations: for heap and index relations.
ExtVacReportType variable helps to determine what the kind is considering
now.
---
src/backend/access/heap/vacuumlazy.c | 99 +++++++++--
src/backend/catalog/system_views.sql | 35 ++++
src/backend/utils/activity/pgstat.c | 7 +-
src/backend/utils/activity/pgstat_relation.c | 41 +++--
src/backend/utils/adt/pgstatfuncs.c | 100 +++++++----
src/include/catalog/pg_proc.dat | 9 +
src/include/pgstat.h | 52 ++++--
.../vacuum-extending-in-repetable-read.out | 7 +-
src/test/regress/expected/rules.out | 25 +++
.../expected/vacuum_index_statistics.out | 165 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 8 +-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 130 ++++++++++++++
.../regress/sql/vacuum_tables_statistics.sql | 3 +-
14 files changed, 601 insertions(+), 81 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d63303c7fb7..9c53d0b4c57 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -168,6 +168,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -246,6 +247,13 @@ typedef struct LVExtStatCounters
PgStat_Counter blocks_hit;
} LVExtStatCounters;
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -408,6 +416,46 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+static void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+static void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -711,14 +759,15 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(rel, &extVacCounters, &extVacReport);
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.pages_frozen = vacrel->set_frozen_pages;
- extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.type = PGSTAT_EXTVAC_HEAP;
+ extVacReport.heap.pages_scanned = vacrel->scanned_pages;
+ extVacReport.heap.pages_removed = vacrel->removed_pages;
+ extVacReport.heap.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.heap.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.heap.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.heap.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.heap.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.heap.index_vacuum_count = vacrel->num_index_scans;
/*
* Report results to the cumulative stats system, too.
@@ -2583,6 +2632,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2601,6 +2654,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2609,6 +2663,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, (void *) vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2633,6 +2694,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2652,12 +2717,20 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3274,7 +3347,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3291,7 +3364,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3307,16 +3380,22 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 4fa9886f409..4da271dae10 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1429,3 +1429,38 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace
LEFT JOIN pg_stat_vacuum_tables(rel.oid) stats ON true
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ COALESCE(total_blks_read, 0) AS total_blks_read,
+ COALESCE(total_blks_hit, 0) AS total_blks_hit,
+ COALESCE(total_blks_dirtied, 0) AS total_blks_dirtied,
+ COALESCE(total_blks_written, 0) AS total_blks_written,
+
+ COALESCE(rel_blks_read, 0) AS rel_blks_read,
+ COALESCE(rel_blks_hit, 0) AS rel_blks_hit,
+
+ COALESCE(pages_deleted, 0) AS pages_deleted,
+ COALESCE(tuples_deleted, 0) AS tuples_deleted,
+
+ COALESCE(wal_records, 0) AS wal_records,
+ COALESCE(wal_fpi, 0) AS wal_fpi,
+ COALESCE(wal_bytes, 0) AS wal_bytes,
+
+ COALESCE(blk_read_time, 0) AS blk_read_time,
+ COALESCE(blk_write_time, 0) AS blk_write_time,
+
+ COALESCE(delay_time, 0) AS delay_time,
+ COALESCE(system_time, 0) AS system_time,
+ COALESCE(user_time, 0) AS user_time,
+ COALESCE(total_time, 0) AS total_time,
+ COALESCE(interrupts, 0) AS interrupts
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace
+ LEFT JOIN pg_stat_vacuum_indexes(rel.oid) stats ON true
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index c283e442f6f..843617eba25 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1122,7 +1122,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
PG_TRY();
{
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
- pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ if (kind == PGSTAT_KIND_RELATION)
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
}
PG_FINALLY();
{
@@ -1177,6 +1178,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 791d777fbc6..5c95363c04a 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -213,7 +213,7 @@ pgstat_drop_relation(Relation rel)
* ---------
*/
void
-pgstat_report_vacuum_error(Oid tableoid)
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -230,6 +230,7 @@ pgstat_report_vacuum_error(Oid tableoid)
tabentry = &shtabentry->stats;
tabentry->vacuum_ext.interrupts++;
+ tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
}
@@ -1042,15 +1043,31 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->pages_frozen += src->pages_frozen;
- dst->pages_all_visible += src->pages_all_visible;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->dead_tuples += src->dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index eba1783e51a..8f5a17e7375 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2101,17 +2101,19 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
}
#define EXTVACHEAPSTAT_COLUMNS 27
+#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
static void
tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
PgStat_StatTabEntry *tabentry)
{
- Datum values[EXTVACHEAPSTAT_COLUMNS];
- bool nulls[EXTVACHEAPSTAT_COLUMNS];
+ Datum values[EXTVACSTAT_COLUMNS];
+ bool nulls[EXTVACSTAT_COLUMNS];
char buf[256];
int i = 0;
- memset(nulls, 0, EXTVACHEAPSTAT_COLUMNS * sizeof(bool));
+ memset(nulls, 0, EXTVACSTAT_COLUMNS * sizeof(bool));
values[i++] = ObjectIdGetDatum(relid);
@@ -2124,16 +2126,25 @@ tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
tabentry->vacuum_ext.blks_hit);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_scanned);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_removed);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.pages_all_visible);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_deleted);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.tuples_frozen);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.dead_tuples);
- values[i++] = Int64GetDatum(tabentry->vacuum_ext.index_vacuum_count);
- values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
- values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+ if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_HEAP)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ }
+ else if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.pages_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.tuples_deleted);
+ }
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
@@ -2161,10 +2172,9 @@ tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
* Get the vacuum statistics for the heap tables or indexes.
*/
static void
-pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
+pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
{
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
- Oid relid = PG_GETARG_OID(0);
PgStat_StatTabEntry *tabentry;
InitMaterializedSRF(fcinfo, 0);
@@ -2177,34 +2187,39 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Assert(rsinfo->setDesc->natts == ncolumns);
Assert(rsinfo->setResult != NULL);
- /* Load table statistics for specified database. */
- if (OidIsValid(relid))
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
{
- tabentry = pgstat_fetch_stat_tabentry(relid);
- if (tabentry == NULL)
- /* Table don't exists or isn't an heap relation. */
- return;
+ Oid relid = PG_GETARG_OID(0);
- tuplestore_put_for_relation(relid, rsinfo, tabentry);
- }
- else
- {
- SnapshotIterator hashiter;
- PgStat_SnapshotEntry *entry;
+ /* Load table statistics for specified relation. */
+ if (OidIsValid(relid))
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ if (tabentry == NULL || tabentry->vacuum_ext.type != type)
+ /* Table don't exists or isn't an heap relation. */
+ return;
+
+ tuplestore_put_for_relation(relid, rsinfo, tabentry);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
- pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
- /* Iterate the snapshot */
- InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
- while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
- {
- CHECK_FOR_INTERRUPTS();
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
- tabentry = (PgStat_StatTabEntry *) entry->data;
+ tabentry = (PgStat_StatTabEntry *) entry->data;
- if (tabentry != NULL)
- tuplestore_put_for_relation(entry->key.objid, rsinfo, tabentry);
+ if (tabentry != NULL && tabentry->vacuum_ext.type == type)
+ tuplestore_put_for_relation(entry->key.objid, rsinfo, tabentry);
+ }
}
}
}
@@ -2215,7 +2230,18 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, int ncolumns)
Datum
pg_stat_vacuum_tables(PG_FUNCTION_ARGS)
{
- pg_stats_vacuum(fcinfo, EXTVACHEAPSTAT_COLUMNS);
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+}
+
+/*
+ * Get the vacuum statistics for the indexes.
+ */
+Datum
+pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
PG_RETURN_VOID();
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 776a7344285..856d04b986f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12372,4 +12372,13 @@
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
prosrc => 'pg_stat_vacuum_tables' },
+{ oid => '8002',
+ descr => 'pg_stat_vacuum_indexes return stats values',
+ proname => 'pg_stat_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_indexes' }
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index e764a8c5326..b784bcc3efe 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,11 +169,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_HEAP = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -205,14 +213,38 @@ typedef struct ExtVacReport
/* Interruptions on any errors. */
int32 interrupts;
- int64 pages_scanned; /* number of pages we examined */
- int64 pages_removed; /* number of pages removed by vacuum */
- int64 pages_frozen; /* number of pages marked in VM as frozen */
- int64 pages_all_visible; /* number of pages marked in VM as all-visible */
- int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
- int64 index_vacuum_count; /* number of index vacuumings */
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } heap;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -694,7 +726,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 7cdb79c0ec4..93fe15c01f9 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -9,10 +9,9 @@ step s2_print_vacuum_stats_table:
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
-relname |tuples_deleted|dead_tuples|tuples_frozen
---------------------------+--------------+-----------+-------------
-test_vacuum_stat_isolation| 0| 0| 0
-(1 row)
+relname|tuples_deleted|dead_tuples|tuples_frozen
+-------+--------------+-----------+-------------
+(0 rows)
step s1_begin_repeatable_read:
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 61df9cfc64c..b2611386211 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2237,6 +2237,31 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ COALESCE(stats.total_blks_read, (0)::bigint) AS total_blks_read,
+ COALESCE(stats.total_blks_hit, (0)::bigint) AS total_blks_hit,
+ COALESCE(stats.total_blks_dirtied, (0)::bigint) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, (0)::bigint) AS total_blks_written,
+ COALESCE(stats.rel_blks_read, (0)::bigint) AS rel_blks_read,
+ COALESCE(stats.rel_blks_hit, (0)::bigint) AS rel_blks_hit,
+ COALESCE(stats.pages_deleted, (0)::bigint) AS pages_deleted,
+ COALESCE(stats.tuples_deleted, (0)::bigint) AS tuples_deleted,
+ COALESCE(stats.wal_records, (0)::bigint) AS wal_records,
+ COALESCE(stats.wal_fpi, (0)::bigint) AS wal_fpi,
+ COALESCE(stats.wal_bytes, (0)::numeric) AS wal_bytes,
+ COALESCE(stats.blk_read_time, (0)::double precision) AS blk_read_time,
+ COALESCE(stats.blk_write_time, (0)::double precision) AS blk_write_time,
+ COALESCE(stats.delay_time, (0)::double precision) AS delay_time,
+ COALESCE(stats.system_time, (0)::double precision) AS system_time,
+ COALESCE(stats.user_time, (0)::double precision) AS user_time,
+ COALESCE(stats.total_time, (0)::double precision) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+ FROM ((pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace)))
+ LEFT JOIN LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON (true))
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..166de176e29
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,165 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_vacuum_indexes(0);
+ min
+------
+ 1232
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index 064064e94b2..8bf706fdf86 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -23,8 +23,6 @@ SELECT pg_stat_force_next_flush();
(1 row)
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -201,9 +199,9 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
(1 row)
SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
- min
------
- 112
+ min
+------
+ 1213
(1 row)
DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 977a0472027..9847a330ed1 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..75e5974eb59
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,130 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+SELECT min(relid) FROM pg_stat_vacuum_indexes(0);
+
+DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
index bc8d051aefa..ed4352566ee 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -17,8 +17,7 @@ SET track_functions TO 'all';
SELECT pg_stat_force_next_flush();
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
+
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
--
2.34.1
[text/x-patch] v10-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (20.9K, 5-v10-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From e8560d296f9440558ae92e8c77518137a3dc9e58 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 22 Oct 2024 21:02:16 +0300
Subject: [PATCH 3/3] Machinery for grabbing an extended vacuum statistics on
databases. It transmits vacuum statistical information about each table and
accumulates it for the database which the table belonged.
---
src/backend/catalog/system_views.sql | 29 ++++++-
src/backend/utils/activity/pgstat.c | 2 +
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 16 ++++
src/backend/utils/adt/pgstatfuncs.c | 77 +++++++++++++++++-
src/include/catalog/pg_proc.dat | 11 ++-
src/include/pgstat.h | 3 +-
src/test/regress/expected/rules.out | 18 +++++
...ut => vacuum_tables_and_db_statistics.out} | 81 ++++++++++++++++++-
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 66 ++++++++++++++-
11 files changed, 291 insertions(+), 15 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (77%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (79%)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 4da271dae10..b68e0f00abd 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1424,7 +1424,6 @@ SELECT
COALESCE(stats.user_time, 0) AS user_time,
COALESCE(stats.total_time, 0) AS total_time,
COALESCE(stats.interrupts, 0) AS interrupts
-
FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace
LEFT JOIN pg_stat_vacuum_tables(rel.oid) stats ON true
@@ -1463,4 +1462,30 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace
LEFT JOIN pg_stat_vacuum_indexes(rel.oid) stats ON true
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ COALESCE(stats.db_blks_read, 0) AS db_blks_read,
+ COALESCE(stats.db_blks_hit, 0) AS db_blks_hit,
+ COALESCE(stats.total_blks_dirtied, 0) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, 0) AS total_blks_written,
+
+ COALESCE(stats.wal_records, 0) AS wal_records,
+ COALESCE(stats.wal_fpi, 0) AS wal_fpi,
+ COALESCE(stats.wal_bytes, 0) AS wal_bytes,
+
+ COALESCE(stats.blk_read_time, 0) AS blk_read_time,
+ COALESCE(stats.blk_write_time, 0) AS blk_write_time,
+
+ COALESCE(stats.delay_time, 0) AS delay_time,
+ COALESCE(stats.system_time, 0) AS system_time,
+ COALESCE(stats.user_time, 0) AS user_time,
+ COALESCE(stats.total_time, 0) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+FROM
+ pg_database db
+ LEFT JOIN pg_stat_vacuum_database(db.oid) stats ON true;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 843617eba25..21b29804620 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1124,6 +1124,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
if (kind == PGSTAT_KIND_RELATION)
pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ else if (kind == PGSTAT_KIND_DATABASE)
+ pgstat_build_snapshot(PGSTAT_KIND_DATABASE);
}
PG_FINALLY();
{
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 29bc0909748..a060d1a4042 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -430,6 +430,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 5c95363c04a..725e26423f2 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -219,6 +219,7 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
if (!pgstat_track_counts)
return;
@@ -232,6 +233,10 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
tabentry->vacuum_ext.interrupts++;
tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.interrupts++;
+ dbentry->vacuum_ext.type = m_type;
}
/*
@@ -245,6 +250,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -298,6 +304,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 8f5a17e7375..cac34fbe64f 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2102,8 +2102,49 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
#define EXTVACHEAPSTAT_COLUMNS 27
#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACDBSTAT_COLUMNS 15
#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
+static void
+tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
+ PgStat_StatDBEntry *dbentry)
+{
+ Datum values[EXTVACDBSTAT_COLUMNS];
+ bool nulls[EXTVACDBSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACDBSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(dbentry->vacuum_ext.interrupts);
+
+ Assert(i == rsinfo->setDesc->natts);
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+}
+
static void
tuplestore_put_for_relation(Oid relid, ReturnSetInfo *rsinfo,
PgStat_StatTabEntry *tabentry)
@@ -2195,8 +2236,9 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
if (OidIsValid(relid))
{
tabentry = pgstat_fetch_stat_tabentry(relid);
- if (tabentry == NULL || tabentry->vacuum_ext.type != type)
- /* Table don't exists or isn't an heap relation. */
+
+ if ((tabentry == NULL || tabentry->vacuum_ext.type != type))
+ /* Table don't exists or isn't a heap or index relation. */
return;
tuplestore_put_for_relation(relid, rsinfo, tabentry);
@@ -2204,7 +2246,7 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
else
{
SnapshotIterator hashiter;
- PgStat_SnapshotEntry *entry;
+ PgStat_SnapshotEntry *entry;
pgstat_update_snapshot(PGSTAT_KIND_RELATION);
@@ -2222,6 +2264,22 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
}
}
}
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ PgStat_StatDBEntry *dbentry;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (OidIsValid(dbid))
+ {
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ /* Database doesn't exist */
+ return;
+
+ tuplestore_put_for_database(dbid, rsinfo, dbentry);
+ }
+ }
}
/*
@@ -2244,4 +2302,15 @@ pg_stat_vacuum_indexes(PG_FUNCTION_ARGS)
pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
PG_RETURN_VOID();
-}
\ No newline at end of file
+}
+
+/*
+ * Get the vacuum statistics for the database.
+ */
+Datum
+pg_stat_vacuum_database(PG_FUNCTION_ARGS)
+{
+ pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
+
+ PG_RETURN_VOID();
+ }
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 856d04b986f..f5f97a80cf5 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12380,5 +12380,14 @@
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
- prosrc => 'pg_stat_vacuum_indexes' }
+ prosrc => 'pg_stat_vacuum_indexes' },
+{ oid => '8003',
+ descr => 'pg_stat_vacuum_database return stats values',
+ proname => 'pg_stat_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stat_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index b784bcc3efe..c6d663c1c48 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -174,7 +174,8 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_HEAP = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index b2611386211..f8112d54f52 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2237,6 +2237,24 @@ pg_stat_user_tables| SELECT relid,
autoanalyze_count
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ COALESCE(stats.db_blks_read, (0)::bigint) AS db_blks_read,
+ COALESCE(stats.db_blks_hit, (0)::bigint) AS db_blks_hit,
+ COALESCE(stats.total_blks_dirtied, (0)::bigint) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, (0)::bigint) AS total_blks_written,
+ COALESCE(stats.wal_records, (0)::bigint) AS wal_records,
+ COALESCE(stats.wal_fpi, (0)::bigint) AS wal_fpi,
+ COALESCE(stats.wal_bytes, (0)::numeric) AS wal_bytes,
+ COALESCE(stats.blk_read_time, (0)::double precision) AS blk_read_time,
+ COALESCE(stats.blk_write_time, (0)::double precision) AS blk_write_time,
+ COALESCE(stats.delay_time, (0)::double precision) AS delay_time,
+ COALESCE(stats.system_time, (0)::double precision) AS system_time,
+ COALESCE(stats.user_time, (0)::double precision) AS user_time,
+ COALESCE(stats.total_time, (0)::double precision) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+ FROM (pg_database db
+ LEFT JOIN LATERAL pg_stat_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON (true));
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 77%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index 8bf706fdf86..7a0b3ba96e1 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,6 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -175,10 +178,7 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
-UPDATE vestat SET x = x1001;
-ERROR: column "x1001" does not exist
-LINE 1: UPDATE vestat SET x = x1001;
- ^
+UPDATE vestat SET x = x+1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
@@ -204,4 +204,77 @@ SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
1213
(1 row)
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | user_time | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 9847a330ed1..1ba32b87cf5 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 79%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index ed4352566ee..a3ddc9419de 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,6 +7,10 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+
-- conditio sine qua non
SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
@@ -140,7 +144,7 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
-UPDATE vestat SET x = x1001;
+UPDATE vestat SET x = x+1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
@@ -156,4 +160,62 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
SELECT min(relid) FROM pg_stat_vacuum_tables(0) where relid > 0;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+DROP TABLE vestat CASCADE;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ user_time > 0 AS user_time,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
--
2.34.1
[text/x-patch] v10-0004-Add-documentation-about-the-system-views-that-are-us.patch (24.2K, 6-v10-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 55a6e8c7dcd9cb3ec9c4e87a13ee9c5bd57183bf Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:47:55 +0300
Subject: [PATCH 4/4] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 747 +++++++++++++++++++++++++++++++++
1 file changed, 747 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 61d28e701f2..93fe9fe36c7 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5064,4 +5064,751 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stats-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this index
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stats-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stats-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-frozen in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_all_visible</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-visible in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this table
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
+
</chapter>
--
2.34.1
[text/plain] vacuum_stats_diff.no-cfbot (23.4K, 7-vacuum_stats_diff.no-cfbot)
download | inline diff:
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index b63c1804b41..b68e0f00abd 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1392,48 +1392,42 @@ SELECT
ns.nspname AS "schema",
rel.relname AS relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
-
- stats.rel_blks_read,
- stats.rel_blks_hit,
-
- stats.pages_scanned,
- stats.pages_removed,
- stats.pages_frozen,
- stats.pages_all_visible,
- stats.tuples_deleted,
- stats.tuples_frozen,
- stats.dead_tuples,
-
- stats.index_vacuum_count,
- stats.rev_all_frozen_pages,
- stats.rev_all_visible_pages,
-
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
-
- stats.blk_read_time,
- stats.blk_write_time,
-
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
- stats.interrupts
-FROM
- pg_database db,
- pg_class rel,
- pg_namespace ns,
- pg_stat_vacuum_tables(rel.oid) stats
-WHERE
- db.datname = current_database() AND
- rel.oid = stats.relid AND
- ns.oid = rel.relnamespace AND
- rel.relkind = 'r';
+ COALESCE(stats.total_blks_read, 0) AS total_blks_read,
+ COALESCE(stats.total_blks_hit, 0) AS total_blks_hit,
+ COALESCE(stats.total_blks_dirtied, 0) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, 0) AS total_blks_written,
+
+ COALESCE(stats.rel_blks_read, 0) AS rel_blks_read,
+ COALESCE(stats.rel_blks_hit, 0) AS rel_blks_hit,
+
+ COALESCE(stats.pages_scanned, 0) AS pages_scanned,
+ COALESCE(stats.pages_removed, 0) AS pages_removed,
+ COALESCE(stats.pages_frozen, 0) AS pages_frozen,
+ COALESCE(stats.pages_all_visible, 0) AS pages_all_visible,
+ COALESCE(stats.tuples_deleted, 0) AS tuples_deleted,
+ COALESCE(stats.tuples_frozen, 0) AS tuples_frozen,
+ COALESCE(stats.dead_tuples, 0) AS dead_tuples,
+
+ COALESCE(stats.index_vacuum_count, 0) AS index_vacuum_count,
+ COALESCE(stats.rev_all_frozen_pages, 0) AS rev_all_frozen_pages,
+ COALESCE(stats.rev_all_visible_pages, 0) AS rev_all_visible_pages,
+
+ COALESCE(stats.wal_records, 0) AS wal_records,
+ COALESCE(stats.wal_fpi, 0) AS wal_fpi,
+ COALESCE(stats.wal_bytes, 0) AS wal_bytes,
+
+ COALESCE(stats.blk_read_time, 0) AS blk_read_time,
+ COALESCE(stats.blk_write_time, 0) AS blk_write_time,
+
+ COALESCE(stats.delay_time, 0) AS delay_time,
+ COALESCE(stats.system_time, 0) AS system_time,
+ COALESCE(stats.user_time, 0) AS user_time,
+ COALESCE(stats.total_time, 0) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace
+ LEFT JOIN pg_stat_vacuum_tables(rel.oid) stats ON true
+WHERE rel.relkind = 'r';
CREATE VIEW pg_stat_vacuum_indexes AS
SELECT
@@ -1441,64 +1435,57 @@ SELECT
ns.nspname AS "schema",
rel.relname AS relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
+ COALESCE(total_blks_read, 0) AS total_blks_read,
+ COALESCE(total_blks_hit, 0) AS total_blks_hit,
+ COALESCE(total_blks_dirtied, 0) AS total_blks_dirtied,
+ COALESCE(total_blks_written, 0) AS total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
+ COALESCE(rel_blks_read, 0) AS rel_blks_read,
+ COALESCE(rel_blks_hit, 0) AS rel_blks_hit,
- stats.pages_deleted,
- stats.tuples_deleted,
+ COALESCE(pages_deleted, 0) AS pages_deleted,
+ COALESCE(tuples_deleted, 0) AS tuples_deleted,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
+ COALESCE(wal_records, 0) AS wal_records,
+ COALESCE(wal_fpi, 0) AS wal_fpi,
+ COALESCE(wal_bytes, 0) AS wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
+ COALESCE(blk_read_time, 0) AS blk_read_time,
+ COALESCE(blk_write_time, 0) AS blk_write_time,
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
- stats.interrupts
+ COALESCE(delay_time, 0) AS delay_time,
+ COALESCE(system_time, 0) AS system_time,
+ COALESCE(user_time, 0) AS user_time,
+ COALESCE(total_time, 0) AS total_time,
+ COALESCE(interrupts, 0) AS interrupts
FROM
- pg_database db,
- pg_class rel,
- pg_namespace ns,
- pg_stat_vacuum_indexes(rel.oid) stats
-WHERE
- db.datname = current_database() AND
- rel.oid = stats.relid AND
- ns.oid = rel.relnamespace AND
- rel.relkind = 'i';
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace
+ LEFT JOIN pg_stat_vacuum_indexes(rel.oid) stats ON true
+WHERE rel.relkind = 'i';
CREATE VIEW pg_stat_vacuum_database AS
SELECT
db.oid as dboid,
db.datname AS dbname,
- stats.db_blks_read,
- stats.db_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
-
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
+ COALESCE(stats.db_blks_read, 0) AS db_blks_read,
+ COALESCE(stats.db_blks_hit, 0) AS db_blks_hit,
+ COALESCE(stats.total_blks_dirtied, 0) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, 0) AS total_blks_written,
- stats.blk_read_time,
- stats.blk_write_time,
+ COALESCE(stats.wal_records, 0) AS wal_records,
+ COALESCE(stats.wal_fpi, 0) AS wal_fpi,
+ COALESCE(stats.wal_bytes, 0) AS wal_bytes,
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
+ COALESCE(stats.blk_read_time, 0) AS blk_read_time,
+ COALESCE(stats.blk_write_time, 0) AS blk_write_time,
- stats.interrupts
+ COALESCE(stats.delay_time, 0) AS delay_time,
+ COALESCE(stats.system_time, 0) AS system_time,
+ COALESCE(stats.user_time, 0) AS user_time,
+ COALESCE(stats.total_time, 0) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
FROM
- pg_database db LEFT JOIN pg_stat_vacuum_database(db.oid) stats
-ON
- db.oid = stats.dboid;
\ No newline at end of file
+ pg_database db
+ LEFT JOIN pg_stat_vacuum_database(db.oid) stats ON true;
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 4d1c099b37e..cac34fbe64f 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2107,7 +2107,7 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
static void
tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
- PgStatShared_Database *dbentry)
+ PgStat_StatDBEntry *dbentry)
{
Datum values[EXTVACDBSTAT_COLUMNS];
bool nulls[EXTVACDBSTAT_COLUMNS];
@@ -2118,28 +2118,28 @@ tuplestore_put_for_database(Oid dbid, ReturnSetInfo *rsinfo,
values[i++] = ObjectIdGetDatum(dbid);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_read);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_hit);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_dirtied);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_written);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.total_blks_written);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_records);
- values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_fpi);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(dbentry->vacuum_ext.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->stats.vacuum_ext.wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->vacuum_ext.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_read_time);
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_write_time);
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.delay_time);
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.system_time);
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.user_time);
- values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.total_time);
- values[i++] = Int32GetDatum(dbentry->stats.vacuum_ext.interrupts);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(dbentry->vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(dbentry->vacuum_ext.interrupts);
Assert(i == rsinfo->setDesc->natts);
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
@@ -2236,8 +2236,9 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
if (OidIsValid(relid))
{
tabentry = pgstat_fetch_stat_tabentry(relid);
- if (tabentry == NULL || tabentry->vacuum_ext.type != type)
- /* Table don't exists or isn't an heap relation. */
+
+ if ((tabentry == NULL || tabentry->vacuum_ext.type != type))
+ /* Table don't exists or isn't a heap or index relation. */
return;
tuplestore_put_for_relation(relid, rsinfo, tabentry);
@@ -2245,7 +2246,7 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
else
{
SnapshotIterator hashiter;
- PgStat_SnapshotEntry *entry;
+ PgStat_SnapshotEntry *entry;
pgstat_update_snapshot(PGSTAT_KIND_RELATION);
@@ -2265,22 +2266,18 @@ pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
}
else if (type == PGSTAT_EXTVAC_DB)
{
- PgStatShared_Database *dbentry;
- PgStat_EntryRef *entry_ref;
- Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ Oid dbid = PG_GETARG_OID(0);
if (OidIsValid(dbid))
{
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dbid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
if (dbentry == NULL)
- /* Table doesn't exist or isn't a heap relation */
+ /* Database doesn't exist */
return;
tuplestore_put_for_database(dbid, rsinfo, dbentry);
- pgstat_unlock_entry(entry_ref);
}
}
}
@@ -2316,4 +2313,4 @@ pg_stat_vacuum_database(PG_FUNCTION_ARGS)
pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
PG_RETURN_VOID();
-}
\ No newline at end of file
+ }
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 8359cf3e984..f8112d54f52 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2239,82 +2239,80 @@ pg_stat_user_tables| SELECT relid,
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_vacuum_database| SELECT db.oid AS dboid,
db.datname AS dbname,
- stats.db_blks_read,
- stats.db_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
- stats.interrupts
+ COALESCE(stats.db_blks_read, (0)::bigint) AS db_blks_read,
+ COALESCE(stats.db_blks_hit, (0)::bigint) AS db_blks_hit,
+ COALESCE(stats.total_blks_dirtied, (0)::bigint) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, (0)::bigint) AS total_blks_written,
+ COALESCE(stats.wal_records, (0)::bigint) AS wal_records,
+ COALESCE(stats.wal_fpi, (0)::bigint) AS wal_fpi,
+ COALESCE(stats.wal_bytes, (0)::numeric) AS wal_bytes,
+ COALESCE(stats.blk_read_time, (0)::double precision) AS blk_read_time,
+ COALESCE(stats.blk_write_time, (0)::double precision) AS blk_write_time,
+ COALESCE(stats.delay_time, (0)::double precision) AS delay_time,
+ COALESCE(stats.system_time, (0)::double precision) AS system_time,
+ COALESCE(stats.user_time, (0)::double precision) AS user_time,
+ COALESCE(stats.total_time, (0)::double precision) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
FROM (pg_database db
- LEFT JOIN LATERAL pg_stat_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON ((db.oid = stats.dboid)));
+ LEFT JOIN LATERAL pg_stat_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON (true));
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_deleted,
- stats.tuples_deleted,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
- stats.interrupts
- FROM pg_database db,
- pg_class rel,
- pg_namespace ns,
- LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
- WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'i'::"char"));
+ COALESCE(stats.total_blks_read, (0)::bigint) AS total_blks_read,
+ COALESCE(stats.total_blks_hit, (0)::bigint) AS total_blks_hit,
+ COALESCE(stats.total_blks_dirtied, (0)::bigint) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, (0)::bigint) AS total_blks_written,
+ COALESCE(stats.rel_blks_read, (0)::bigint) AS rel_blks_read,
+ COALESCE(stats.rel_blks_hit, (0)::bigint) AS rel_blks_hit,
+ COALESCE(stats.pages_deleted, (0)::bigint) AS pages_deleted,
+ COALESCE(stats.tuples_deleted, (0)::bigint) AS tuples_deleted,
+ COALESCE(stats.wal_records, (0)::bigint) AS wal_records,
+ COALESCE(stats.wal_fpi, (0)::bigint) AS wal_fpi,
+ COALESCE(stats.wal_bytes, (0)::numeric) AS wal_bytes,
+ COALESCE(stats.blk_read_time, (0)::double precision) AS blk_read_time,
+ COALESCE(stats.blk_write_time, (0)::double precision) AS blk_write_time,
+ COALESCE(stats.delay_time, (0)::double precision) AS delay_time,
+ COALESCE(stats.system_time, (0)::double precision) AS system_time,
+ COALESCE(stats.user_time, (0)::double precision) AS user_time,
+ COALESCE(stats.total_time, (0)::double precision) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+ FROM ((pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace)))
+ LEFT JOIN LATERAL pg_stat_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON (true))
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_scanned,
- stats.pages_removed,
- stats.pages_frozen,
- stats.pages_all_visible,
- stats.tuples_deleted,
- stats.tuples_frozen,
- stats.dead_tuples,
- stats.index_vacuum_count,
- stats.rev_all_frozen_pages,
- stats.rev_all_visible_pages,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.system_time,
- stats.user_time,
- stats.total_time,
- stats.interrupts
- FROM pg_database db,
- pg_class rel,
- pg_namespace ns,
- LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
- WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace) AND (rel.relkind = 'r'::"char"));
+ COALESCE(stats.total_blks_read, (0)::bigint) AS total_blks_read,
+ COALESCE(stats.total_blks_hit, (0)::bigint) AS total_blks_hit,
+ COALESCE(stats.total_blks_dirtied, (0)::bigint) AS total_blks_dirtied,
+ COALESCE(stats.total_blks_written, (0)::bigint) AS total_blks_written,
+ COALESCE(stats.rel_blks_read, (0)::bigint) AS rel_blks_read,
+ COALESCE(stats.rel_blks_hit, (0)::bigint) AS rel_blks_hit,
+ COALESCE(stats.pages_scanned, (0)::bigint) AS pages_scanned,
+ COALESCE(stats.pages_removed, (0)::bigint) AS pages_removed,
+ COALESCE(stats.pages_frozen, (0)::bigint) AS pages_frozen,
+ COALESCE(stats.pages_all_visible, (0)::bigint) AS pages_all_visible,
+ COALESCE(stats.tuples_deleted, (0)::bigint) AS tuples_deleted,
+ COALESCE(stats.tuples_frozen, (0)::bigint) AS tuples_frozen,
+ COALESCE(stats.dead_tuples, (0)::bigint) AS dead_tuples,
+ COALESCE(stats.index_vacuum_count, (0)::bigint) AS index_vacuum_count,
+ COALESCE(stats.rev_all_frozen_pages, (0)::bigint) AS rev_all_frozen_pages,
+ COALESCE(stats.rev_all_visible_pages, (0)::bigint) AS rev_all_visible_pages,
+ COALESCE(stats.wal_records, (0)::bigint) AS wal_records,
+ COALESCE(stats.wal_fpi, (0)::bigint) AS wal_fpi,
+ COALESCE(stats.wal_bytes, (0)::numeric) AS wal_bytes,
+ COALESCE(stats.blk_read_time, (0)::double precision) AS blk_read_time,
+ COALESCE(stats.blk_write_time, (0)::double precision) AS blk_write_time,
+ COALESCE(stats.delay_time, (0)::double precision) AS delay_time,
+ COALESCE(stats.system_time, (0)::double precision) AS system_time,
+ COALESCE(stats.user_time, (0)::double precision) AS user_time,
+ COALESCE(stats.total_time, (0)::double precision) AS total_time,
+ COALESCE(stats.interrupts, 0) AS interrupts
+ FROM ((pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace)))
+ LEFT JOIN LATERAL pg_stat_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts) ON (true))
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index 4f6e305710e..166de176e29 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -35,9 +35,10 @@ DELETE FROM vestat WHERE x % 2 = 0;
SELECT vt.relname,relpages,pages_deleted,tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
----------+----------+---------------+----------------
-(0 rows)
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
SELECT relpages AS irp
FROM pg_class c
diff --git a/src/test/regress/expected/vacuum_tables_and_db_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
index 94dd3214349..7a0b3ba96e1 100644
--- a/src/test/regress/expected/vacuum_tables_and_db_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -26,8 +26,6 @@ SELECT pg_stat_force_next_flush();
(1 row)
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -40,7 +38,8 @@ FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
---------+--------------+----------------+----------+---------------+---------------
-(0 rows)
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
SELECT relpages AS rp
FROM pg_class c
@@ -179,10 +178,7 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
-UPDATE vestat SET x = x1001;
-ERROR: column "x1001" does not exist
-LINE 1: UPDATE vestat SET x = x1001;
- ^
+UPDATE vestat SET x = x+1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
@@ -259,8 +255,6 @@ WHERE dbname = 'regression_statistic_vacuum_db';
(1 row)
\c regression_statistic_vacuum_db
-RESET vacuum_freeze_min_age;
-RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
\c regression_statistic_vacuum_db1;
SELECT count(*)
diff --git a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index af1281b3b63..a3ddc9419de 100644
--- a/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -21,8 +21,7 @@ SET track_functions TO 'all';
SELECT pg_stat_force_next_flush();
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
+
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -145,7 +144,7 @@ FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
-UPDATE vestat SET x = x1001;
+UPDATE vestat SET x = x+1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
@@ -204,8 +203,6 @@ WHERE dbname = 'regression_statistic_vacuum_db';
\c regression_statistic_vacuum_db
-RESET vacuum_freeze_min_age;
-RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
\c regression_statistic_vacuum_db1;
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-10-28 13:40 Alexander Korotkov <[email protected]>
parent: Alena Rybakina <[email protected]>
2 siblings, 2 replies; 77+ messages in thread
From: Alexander Korotkov @ 2024-10-28 13:40 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]; jian he <[email protected]>
On Sun, Aug 25, 2024 at 6:59 PM Alena Rybakina
<[email protected]> wrote:
> I didn't understand correctly - did you mean that we don't need SRF if
> we need to display statistics for a specific object?
>
> Otherwise, we need this when we display information on all database
> objects (tables or indexes):
>
> while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter))
> != NULL)
> {
> CHECK_FOR_INTERRUPTS();
>
> tabentry = (PgStat_StatTabEntry *) entry->data;
>
> if (tabentry != NULL && tabentry->vacuum_ext.type == type)
> tuplestore_put_for_relation(relid, rsinfo, tabentry);
> }
>
> I know we can construct a HeapTuple object containing a TupleDesc,
> values, and nulls for a particular object, but I'm not sure we can
> augment it while looping through multiple objects.
>
> /* Initialise attributes information in the tuple descriptor */
>
> tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_STATS_COLS);
>
> ...
>
> PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
>
>
> If I missed something or misunderstood, can you explain in more detail?
Actually, I mean why do we need a possibility to return statistics for
all tables/indexes in one function call? User anyway is supposed to
use pg_stat_vacuum_indexes/pg_stat_vacuum_tables view, which do
function calls one per relation. I suppose we can get rid of
possibility to get all the objects in one function call and just
return a tuple from the functions like other pgstatfuncs.c functions
do.
------
Regards,
Alexander Korotkov
Supabase
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-10-29 11:02 Alena Rybakina <[email protected]>
parent: Alexander Korotkov <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2024-10-29 11:02 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]; jian he <[email protected]>
On 28.10.2024 16:40, Alexander Korotkov wrote:
> On Sun, Aug 25, 2024 at 6:59 PM Alena Rybakina
> <[email protected]> wrote:
>> I didn't understand correctly - did you mean that we don't need SRF if
>> we need to display statistics for a specific object?
>>
>> Otherwise, we need this when we display information on all database
>> objects (tables or indexes):
>>
>> while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter))
>> != NULL)
>> {
>> CHECK_FOR_INTERRUPTS();
>>
>> tabentry = (PgStat_StatTabEntry *) entry->data;
>>
>> if (tabentry != NULL && tabentry->vacuum_ext.type == type)
>> tuplestore_put_for_relation(relid, rsinfo, tabentry);
>> }
>>
>> I know we can construct a HeapTuple object containing a TupleDesc,
>> values, and nulls for a particular object, but I'm not sure we can
>> augment it while looping through multiple objects.
>>
>> /* Initialise attributes information in the tuple descriptor */
>>
>> tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_STATS_COLS);
>>
>> ...
>>
>> PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
>>
>>
>> If I missed something or misunderstood, can you explain in more detail?
> Actually, I mean why do we need a possibility to return statistics for
> all tables/indexes in one function call? User anyway is supposed to
> use pg_stat_vacuum_indexes/pg_stat_vacuum_tables view, which do
> function calls one per relation. I suppose we can get rid of
> possibility to get all the objects in one function call and just
> return a tuple from the functions like other pgstatfuncs.c functions
> do.
>
I haven’t thought about this before and agree with you. Thanks for the
clarification! I'll fix the patch this evening and release the updated
version.
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-11-02 12:24 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2024-11-02 12:24 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; pgsql-hackers; [email protected]; jian he <[email protected]>
Hi!
On 29.10.2024 14:02, Alena Rybakina wrote:
> On 28.10.2024 16:40, Alexander Korotkov wrote:
>> On Sun, Aug 25, 2024 at 6:59 PM Alena Rybakina
>> <[email protected]> wrote:
>>> I didn't understand correctly - did you mean that we don't need SRF if
>>> we need to display statistics for a specific object?
>>>
>>> Otherwise, we need this when we display information on all database
>>> objects (tables or indexes):
>>>
>>> while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter))
>>> != NULL)
>>> {
>>> CHECK_FOR_INTERRUPTS();
>>>
>>> tabentry = (PgStat_StatTabEntry *) entry->data;
>>>
>>> if (tabentry != NULL && tabentry->vacuum_ext.type == type)
>>> tuplestore_put_for_relation(relid, rsinfo, tabentry);
>>> }
>>>
>>> I know we can construct a HeapTuple object containing a TupleDesc,
>>> values, and nulls for a particular object, but I'm not sure we can
>>> augment it while looping through multiple objects.
>>>
>>> /* Initialise attributes information in the tuple descriptor */
>>>
>>> tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_STATS_COLS);
>>>
>>> ...
>>>
>>> PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
>>>
>>>
>>> If I missed something or misunderstood, can you explain in more detail?
>> Actually, I mean why do we need a possibility to return statistics for
>> all tables/indexes in one function call? User anyway is supposed to
>> use pg_stat_vacuum_indexes/pg_stat_vacuum_tables view, which do
>> function calls one per relation. I suppose we can get rid of
>> possibility to get all the objects in one function call and just
>> return a tuple from the functions like other pgstatfuncs.c functions
>> do.
>>
> I haven’t thought about this before and agree with you. Thanks for the
> clarification! I'll fix the patch this evening and release the updated
> version.
I updated the patches as per your suggestion. You can see it here [0].
[0]
https://www.postgresql.org/message-id/85b963fe-5977-43aa-9241-75b862abcc69%40postgrespro.ru
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2024-11-07 14:49 Ilia Evdokimov <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Ilia Evdokimov @ 2024-11-07 14:49 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; pgsql-hackers; +Cc: jian he <[email protected]>; Alexander Korotkov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; [email protected]
On 22.10.2024 22:30, Alena Rybakina wrote:
>
> Hi!
>
> On 16.10.2024 14:01, Alena Rybakina wrote:
>>>
>>> Thank you for rebasing.
>>>
>>> I have noticed that when I create a table or an index on this table,
>>> there is no information about the table or index in
>>> pg_stat_vacuum_tables and pg_stat_vacuum_indexes until we perform a
>>> VACUUM.
>>>
>>> Example:
>>>
>>> CREATE TABLE t (i INT, j INT);
>>> INSERT INTO t SELECT i/10, i/100 FROM GENERATE_SERIES(1,1000000) i;
>>> SELECT * FROM pg_stat_vacuum_tables WHERE relname = 't';
>>> ....
>>> (0 rows)
>>> CREATE INDEX ON t (i);
>>> SELECT * FROM pg_stat_vacuum_indexes WHERE relname = 't_i_idx';
>>> ...
>>> (0 rows)
>>>
>>> I can see the entries after running VACUUM or executing
>>> autovacuum. or when autovacuum is executed. I would suggest adding a
>>> line about the relation even if it has not yet been processed by
>>> vacuum. Interestingly, this issue does not occur with
>>> pg_stat_vacuum_database:
>>>
>>> CREATE DATABASE example_db;
>>> SELECT * FROM pg_stat_vacuum_database WHERE dbname = 'example_db';
>>> dboid | dbname | ...
>>> ... | example_db | ...
>>> (1 row)
>>>
>>> BTW, I recommend renaming the view pg_stat_vacuum_database to
>>> pg_stat_vacuum_database_S_ for consistency with
>>> pg_stat_vacuum_tables and pg_stat_vacuum_indexes
>>>
>> Thanks for the review. I'm investigating this. I agree with the
>> renaming, I will do it in the next version of the patch.
>>
> I fixed it. I added the left outer join to the vacuum views and for
> converting the coalesce function from NULL to null values.
>
> I also fixed the code in getting database statistics - we can get it
> through the existing pgstat_fetch_stat_dbentry function and fixed
> couple of comments.
>
> I attached a diff file, as well as new versions of patches.
>
> --
> Regards,
> Alena Rybakina
> Postgres Professional
Thank you for fixing it.
1) I have found some typos in the test output files (out-files) when
running 'make check' and 'make check-world'. These typos might cause
minor discrepancies in test results. You may already be aware of them,
but I wanted to bring them to your attention in case they haven't been
noticed. I believe these can be fixed quickly.
2) Additionally, I observed that when we create a table and insert some
rows, executing the VACUUM FULL command does not update the information
in the 'pg_stat_get_vacuum_tables' However, running the VACUUM command
does update this information as expected. This seems inconsistent, and
it might be a bug.
Example:
CREATE TABLE t (i INT, j INT) WITH (autovacuum_enabled = false);
INSERT INTO t SELECT i/10, i/100 FROM GENERATE_SERIES(1,1000000) i;
SELECT * FROM pg_stat_get_vacuum_tables WHERE relname = 't';
schema | relname | relid | total_blks_read | .........
-----------+------------+---------+----------------------+---------
public | t | 21416 | 0 | ......
(1 row)
VACUUM FULL;
SELECT * FROM pg_stat_get_vacuum_tables WHERE relname = 't';
schema | relname | relid | total_blks_read | .........
-----------+------------+---------+----------------------+---------
public | t | 21416 | 0 | ......
(1 row)
VACUUM;
SELECT * FROM pg_stat_get_vacuum_tables WHERE relname = 't';
schema | relname | relid | total_blks_read | .........
-----------+------------+---------+----------------------+---------
public | t | 21416 | 4425 | ......
(1 row)
Regards,
Ilia Evdokimov,
Tantor Labs LLC.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-10 09:13 Ilia Evdokimov <[email protected]>
3 siblings, 1 reply; 77+ messages in thread
From: Ilia Evdokimov @ 2025-03-10 09:13 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Jim Nasby <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
Hi,
After commit eaf5027 we should add information about wal_buffers_full.
Any thoughts?
--
Best regards,
Ilia Evdokimov,
Tantor Labs LLC.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-10 13:33 Kirill Reshke <[email protected]>
3 siblings, 1 reply; 77+ messages in thread
From: Kirill Reshke @ 2025-03-10 13:33 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: pgsql-hackers; Alexander Korotkov <[email protected]>; Jim Nasby <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
On Thu, 27 Feb 2025 at 23:00, Alena Rybakina <[email protected]> wrote:
>
> Hi!
> On 17.02.2025 17:46, Alena Rybakina wrote:
> > On 04.02.2025 18:22, Alena Rybakina wrote:
> >> Hi! Thank you for your review!
> >>
> >> On 02.02.2025 23:43, Alexander Korotkov wrote:
> >>> On Mon, Jan 13, 2025 at 3:26 PM Alena Rybakina
> >>> <[email protected]> wrote:
> >>>> I noticed that the cfbot is bad, the reason seems to be related to
> >>>> the lack of a parameter in
> >>>> src/backend/utils/misc/postgresql.conf.sample. I added it, it
> >>>> should help.
> >>> The patch doesn't apply cleanly. Please rebase.
> >> I rebased them.
> > The patch needed a rebase again. There is nothing new since version
> > 18, only a rebase.
>
> The patch needed a new rebase.
>
> Sorry, but due to feeling unwell I picked up a patch from another thread
> and squashed the patch for vacuum statistics for indexes and heaps in
> the previous version. Now I fixed everything together with the rebase.
>
> --
> Regards,
> Alena Rybakina
> Postgres Professional
Hi!
CI fails on this one[0]
Is it a flap or a real problem caused by v20?
```
SELECT relpages AS irp
diff -U3 /tmp/cirrus-ci-build/src/test/regress/expected/vacuum_tables_and_db_statistics.out
/tmp/cirrus-ci-build/build-32/testrun/recovery/027_stream_regress/data/results/vacuum_tables_and_db_statistics.out
--- /tmp/cirrus-ci-build/src/test/regress/expected/vacuum_tables_and_db_statistics.out
2025-03-10 09:36:10.274799176 +0000
+++ /tmp/cirrus-ci-build/build-32/testrun/recovery/027_stream_regress/data/results/vacuum_tables_and_db_statistics.out
2025-03-10 09:49:35.796596462 +0000
@@ -65,7 +65,7 @@
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
relname | vm_new_frozen_pages | tuples_deleted | relpages |
pages_scanned | pages_removed
---------+---------------------+----------------+----------+---------------+---------------
- vestat | 0 | 0 | 455 |
0 | 0
+ vestat | 0 | 0 | 417 |
0 | 0
(1 row)
SELECT relpages AS rp
=== EOF ===
```
[0] https://api.cirrus-ci.com/v1/artifact/task/5336493629112320/testrun/build-32/testrun/recovery/027_st...
--
Best regards,
Kirill Reshke
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-12 19:36 Alena Rybakina <[email protected]>
parent: Kirill Reshke <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2025-03-12 19:36 UTC (permalink / raw)
To: Kirill Reshke <[email protected]>; +Cc: pgsql-hackers; Alexander Korotkov <[email protected]>; Jim Nasby <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
Hi!
On 10.03.2025 16:33, Kirill Reshke wrote:
> On Thu, 27 Feb 2025 at 23:00, Alena Rybakina<[email protected]> wrote:
>> Hi!
>> On 17.02.2025 17:46, Alena Rybakina wrote:
>>> On 04.02.2025 18:22, Alena Rybakina wrote:
>>>> Hi! Thank you for your review!
>>>>
>>>> On 02.02.2025 23:43, Alexander Korotkov wrote:
>>>>> On Mon, Jan 13, 2025 at 3:26 PM Alena Rybakina
>>>>> <[email protected]> wrote:
>>>>>> I noticed that the cfbot is bad, the reason seems to be related to
>>>>>> the lack of a parameter in
>>>>>> src/backend/utils/misc/postgresql.conf.sample. I added it, it
>>>>>> should help.
>>>>> The patch doesn't apply cleanly. Please rebase.
>>>> I rebased them.
>>> The patch needed a rebase again. There is nothing new since version
>>> 18, only a rebase.
>> The patch needed a new rebase.
>>
>> Sorry, but due to feeling unwell I picked up a patch from another thread
>> and squashed the patch for vacuum statistics for indexes and heaps in
>> the previous version. Now I fixed everything together with the rebase.
>>
>> --
>> Regards,
>> Alena Rybakina
>> Postgres Professional
> Hi!
> CI fails on this one[0]
>
> Is it a flap or a real problem caused by v20?
>
> ```
>
> SELECT relpages AS irp
> diff -U3 /tmp/cirrus-ci-build/src/test/regress/expected/vacuum_tables_and_db_statistics.out
> /tmp/cirrus-ci-build/build-32/testrun/recovery/027_stream_regress/data/results/vacuum_tables_and_db_statistics.out
> --- /tmp/cirrus-ci-build/src/test/regress/expected/vacuum_tables_and_db_statistics.out
> 2025-03-10 09:36:10.274799176 +0000
> +++ /tmp/cirrus-ci-build/build-32/testrun/recovery/027_stream_regress/data/results/vacuum_tables_and_db_statistics.out
> 2025-03-10 09:49:35.796596462 +0000
> @@ -65,7 +65,7 @@
> WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
> relname | vm_new_frozen_pages | tuples_deleted | relpages |
> pages_scanned | pages_removed
> ---------+---------------------+----------------+----------+---------------+---------------
> - vestat | 0 | 0 | 455 |
> 0 | 0
> + vestat | 0 | 0 | 417 |
> 0 | 0
> (1 row)
>
> SELECT relpages AS rp
> === EOF ===
>
>
> ```
>
> [0]https://api.cirrus-ci.com/v1/artifact/task/5336493629112320/testrun/build-32/testrun/recovery/027_st...
Thank you for your help, I'll fix it soon.
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-12 19:41 Alena Rybakina <[email protected]>
parent: Ilia Evdokimov <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2025-03-12 19:41 UTC (permalink / raw)
To: Ilia Evdokimov <[email protected]>; +Cc: pgsql-hackers; Alexander Korotkov <[email protected]>; Jim Nasby <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
Hi!
On 10.03.2025 12:13, Ilia Evdokimov wrote:
> Hi,
>
> After commit eaf5027 we should add information about wal_buffers_full.
>
> Any thoughts?
>
> --
> Best regards,
> Ilia Evdokimov,
> Tantor Labs LLC.
>
I think I can add it. To be honest, I haven't gotten to know this
statistic yet, haven't had time to get to know this commit yet. How will
this statistic help us analyze the work of the vacuum for relations?
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-12 22:15 Jim Nasby <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Jim Nasby @ 2025-03-12 22:15 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
On Wed, Mar 12, 2025 at 2:41 PM Alena Rybakina <[email protected]>
wrote:
> Hi!
>
> On 10.03.2025 12:13, Ilia Evdokimov wrote:
> > Hi,
> >
> > After commit eaf5027 we should add information about wal_buffers_full.
> >
> > Any thoughts?
> >
> > --
> > Best regards,
> > Ilia Evdokimov,
> > Tantor Labs LLC.
> >
> I think I can add it. To be honest, I haven't gotten to know this
> statistic yet, haven't had time to get to know this commit yet. How will
> this statistic help us analyze the work of the vacuum for relations?
>
The usecase I can see here is that we don't want autovac creating so much
WAL traffic that it starts forcing other backends to have to write WAL out.
But tracking how many times autovac writes WAL buffers won't help with that
(though we also don't want any WAL buffers written by autovac to be counted
in the system-wide wal_buffers_full: autovac is a BG process and it's fine
if it's spending time writing WAL out). I think the same is true of a
manual vacuum as well.
What would be helpful would be a way to determine if autovac was causing
enough traffic to force other backends to write WAL. Offhand I'm not sure
how practical that actually is though.
BTW, there's also an argument to be made that autovac should throttle
itself if we're close to running out of available WAL buffers...
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-13 06:42 Bertrand Drouvot <[email protected]>
parent: Jim Nasby <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Bertrand Drouvot @ 2025-03-13 06:42 UTC (permalink / raw)
To: Jim Nasby <[email protected]>; +Cc: Alena Rybakina <[email protected]>; Ilia Evdokimov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
Hi,
On Wed, Mar 12, 2025 at 05:15:53PM -0500, Jim Nasby wrote:
> The usecase I can see here is that we don't want autovac creating so much
> WAL traffic that it starts forcing other backends to have to write WAL out.
> But tracking how many times autovac writes WAL buffers won't help with that
Right, because the one that increments the wal_buffers_full metric could "just"
be a victim (i.e the one that happens to trigger the WAL buffers disk flush,
even though other backends contributed most of the buffer usage).
> (though we also don't want any WAL buffers written by autovac to be counted
> in the system-wide wal_buffers_full:
why? Or do you mean that it would be good to have 2 kinds of metrics: one
generated by "maintenance" activity and one by "regular" backends?
> What would be helpful would be a way to determine if autovac was causing
> enough traffic to force other backends to write WAL. Offhand I'm not sure
> how practical that actually is though.
a051e71e28a could help to see how much WAL has by written by the autovac workers.
> BTW, there's also an argument to be made that autovac should throttle
> itself if we're close to running out of available WAL buffers...
hmm, yeah I think that's an interesting idea OTOH that would mean to "delegate"
the WAL buffers flush to another backend.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-17 06:42 vignesh C <[email protected]>
3 siblings, 1 reply; 77+ messages in thread
From: vignesh C @ 2025-03-17 06:42 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: pgsql-hackers; Alexander Korotkov <[email protected]>; Jim Nasby <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
On Thu, 27 Feb 2025 at 23:30, Alena Rybakina <[email protected]> wrote:
>
> Hi!
> On 17.02.2025 17:46, Alena Rybakina wrote:
> > On 04.02.2025 18:22, Alena Rybakina wrote:
> >> Hi! Thank you for your review!
> >>
> >> On 02.02.2025 23:43, Alexander Korotkov wrote:
> >>> On Mon, Jan 13, 2025 at 3:26 PM Alena Rybakina
> >>> <[email protected]> wrote:
> >>>> I noticed that the cfbot is bad, the reason seems to be related to
> >>>> the lack of a parameter in
> >>>> src/backend/utils/misc/postgresql.conf.sample. I added it, it
> >>>> should help.
> >>> The patch doesn't apply cleanly. Please rebase.
> >> I rebased them.
> > The patch needed a rebase again. There is nothing new since version
> > 18, only a rebase.
>
> The patch needed a new rebase.
I noticed that the CI failure reported at [1], Ilia's comment from
[2], changed the status to Waiting on Author, please address them and
update it to Needs review.
[1] - https://www.postgresql.org/message-id/CALdSSPiw_-0_L3YV%3DQn7oopPqY2XVrXwDSGLdSXS69QvMdXisQ%40mail.g...
[2] - https://www.postgresql.org/message-id/47a7b784-5218-43f2-96e3-65f9a729c5a5%40tantorlabs.com
Regards,
Vignesh
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-18 05:57 Alena Rybakina <[email protected]>
parent: vignesh C <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2025-03-18 05:57 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: pgsql-hackers; Alexander Korotkov <[email protected]>; Jim Nasby <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
On 17.03.2025 09:42, vignesh C wrote:
> On Thu, 27 Feb 2025 at 23:30, Alena Rybakina<[email protected]> wrote:
>> Hi!
>> On 17.02.2025 17:46, Alena Rybakina wrote:
>>> On 04.02.2025 18:22, Alena Rybakina wrote:
>>>> Hi! Thank you for your review!
>>>>
>>>> On 02.02.2025 23:43, Alexander Korotkov wrote:
>>>>> On Mon, Jan 13, 2025 at 3:26 PM Alena Rybakina
>>>>> <[email protected]> wrote:
>>>>>> I noticed that the cfbot is bad, the reason seems to be related to
>>>>>> the lack of a parameter in
>>>>>> src/backend/utils/misc/postgresql.conf.sample. I added it, it
>>>>>> should help.
>>>>> The patch doesn't apply cleanly. Please rebase.
>>>> I rebased them.
>>> The patch needed a rebase again. There is nothing new since version
>>> 18, only a rebase.
>> The patch needed a new rebase.
> I noticed that the CI failure reported at [1], Ilia's comment from
> [2], changed the status to Waiting on Author, please address them and
> update it to Needs review.
> [1] -https://www.postgresql.org/message-id/CALdSSPiw_-0_L3YV%3DQn7oopPqY2XVrXwDSGLdSXS69QvMdXisQ%40mail.g...
> [2] -https://www.postgresql.org/message-id/47a7b784-5218-43f2-96e3-65f9a729c5a5%40tantorlabs.com
Okay, thank you!
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-21 19:42 Alena Rybakina <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 3 replies; 77+ messages in thread
From: Alena Rybakina @ 2025-03-21 19:42 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; Jim Nasby <[email protected]>; vignesh C <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
On 13.03.2025 09:42, Bertrand Drouvot wrote:
> Hi,
>
> On Wed, Mar 12, 2025 at 05:15:53PM -0500, Jim Nasby wrote:
>> The usecase I can see here is that we don't want autovac creating so much
>> WAL traffic that it starts forcing other backends to have to write WAL out.
>> But tracking how many times autovac writes WAL buffers won't help with that
> Right, because the one that increments the wal_buffers_full metric could "just"
> be a victim (i.e the one that happens to trigger the WAL buffers disk flush,
> even though other backends contributed most of the buffer usage).
>
>> (though we also don't want any WAL buffers written by autovac to be counted
>> in the system-wide wal_buffers_full:
> why? Or do you mean that it would be good to have 2 kinds of metrics: one
> generated by "maintenance" activity and one by "regular" backends?
>
>> What would be helpful would be a way to determine if autovac was causing
>> enough traffic to force other backends to write WAL. Offhand I'm not sure
>> how practical that actually is though.
> a051e71e28a could help to see how much WAL has by written by the autovac workers.
>
>> BTW, there's also an argument to be made that autovac should throttle
>> itself if we're close to running out of available WAL buffers...
> hmm, yeah I think that's an interesting idea OTOH that would mean to "delegate"
> the WAL buffers flush to another backend.
>
> Regards,
>
I will add it and fix the tests but later and I'll explain why.
I'm working on this issue [0] and try have already created new
statistics in Statistics Collector to store database and relation vacuum
statistics: PGSTAT_KIND_VACUUM_DB and PGSTAT_KIND_VACUUM_RELATION.
Vacuum statistics are saved there instead of relation's and database's
statistic structure, but for some reason it is not possible to find them
in the hash table when building a snapshot and display them accordingly.
I have not yet figured out where the error is.
Without solving this problem, committing vacuum statistics is not yet
possible. An alternative way for us was to refuse some statistics for
now for relations,
but we could not agree on which statistics should not be displayed yet
and for now we are only adding them :).
I understand why this is important to display more vacuum information
about vacuum statistics - it will allow us to better understand the
problems of incorrect vacuum settings or, for example, notice a bug in
its operation.
In order to reduce the memory consumption for storing them for those who
are not going to use them, I just realized that we need to create a
separate space for storing the statistics
I mentioned above (PGSTAT_KIND_VACUUM_DB and
PGSTAT_KIND_VACUUM_RELATION), there is no other way to do this and I am
still trying to complete this functionality.
I doubt that I will have time for this by code freeze date and even if I
do, I will hardly have time for a normal review. There's really a lot
more to learn related to the stat collector, so
I'm postponing it to the next commitfest.
Sorry. I'll fix the tests as soon as I finish this part, since they'll
most likely either break the same way or in some new way.
Tomorrow or the day after tomorrow I will send a diff patch with what I
have already managed to demonstrate the problem, since I need to bring
the code to a normal form.
Maybe someone who worked with the stat collector will suddenly tell me
where and what I have implemented incorrectly.
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-21 19:46 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
2 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2025-03-21 19:46 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; Jim Nasby <[email protected]>; vignesh C <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
Sorry, I forgot to provide a link to the problem [0], actually. So I
provided it below.
[0]
https://www.postgresql.org/message-id/CAPpHfduoJEuoixPTTg2tjhnXqrdobuMaQGxriqxJ9TjN1uxOuA%40mail.gma...
On 21.03.2025 22:42, Alena Rybakina wrote:
> On 13.03.2025 09:42, Bertrand Drouvot wrote:
>> Hi,
>>
>> On Wed, Mar 12, 2025 at 05:15:53PM -0500, Jim Nasby wrote:
>>> The usecase I can see here is that we don't want autovac creating so much
>>> WAL traffic that it starts forcing other backends to have to write WAL out.
>>> But tracking how many times autovac writes WAL buffers won't help with that
>> Right, because the one that increments the wal_buffers_full metric could "just"
>> be a victim (i.e the one that happens to trigger the WAL buffers disk flush,
>> even though other backends contributed most of the buffer usage).
>>
>>> (though we also don't want any WAL buffers written by autovac to be counted
>>> in the system-wide wal_buffers_full:
>> why? Or do you mean that it would be good to have 2 kinds of metrics: one
>> generated by "maintenance" activity and one by "regular" backends?
>>
>>> What would be helpful would be a way to determine if autovac was causing
>>> enough traffic to force other backends to write WAL. Offhand I'm not sure
>>> how practical that actually is though.
>> a051e71e28a could help to see how much WAL has by written by the autovac workers.
>>
>>> BTW, there's also an argument to be made that autovac should throttle
>>> itself if we're close to running out of available WAL buffers...
>> hmm, yeah I think that's an interesting idea OTOH that would mean to "delegate"
>> the WAL buffers flush to another backend.
>>
>> Regards,
>>
>
> I will add it and fix the tests but later and I'll explain why.
>
> I'm working on this issue [0] and try have already created new
> statistics in Statistics Collector to store database and relation
> vacuum statistics: PGSTAT_KIND_VACUUM_DB and PGSTAT_KIND_VACUUM_RELATION.
>
> Vacuum statistics are saved there instead of relation's and database's
> statistic structure, but for some reason it is not possible to find
> them in the hash table when building a snapshot and display them
> accordingly.
> I have not yet figured out where the error is.
>
> Without solving this problem, committing vacuum statistics is not yet
> possible. An alternative way for us was to refuse some statistics for
> now for relations,
> but we could not agree on which statistics should not be displayed yet
> and for now we are only adding them :).
>
> I understand why this is important to display more vacuum information
> about vacuum statistics - it will allow us to better understand the
> problems of incorrect vacuum settings or, for example, notice a bug in
> its operation.
>
> In order to reduce the memory consumption for storing them for those
> who are not going to use them, I just realized that we need to create
> a separate space for storing the statistics
> I mentioned above (PGSTAT_KIND_VACUUM_DB and
> PGSTAT_KIND_VACUUM_RELATION), there is no other way to do this and I
> am still trying to complete this functionality.
>
> I doubt that I will have time for this by code freeze date and even if
> I do, I will hardly have time for a normal review. There's really a
> lot more to learn related to the stat collector, so
> I'm postponing it to the next commitfest.
>
> Sorry. I'll fix the tests as soon as I finish this part, since they'll
> most likely either break the same way or in some new way.
>
> Tomorrow or the day after tomorrow I will send a diff patch with what
> I have already managed to demonstrate the problem, since I need to
> bring the code to a normal form.
> Maybe someone who worked with the stat collector will suddenly tell me
> where and what I have implemented incorrectly.
>
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-24 23:02 Jim Nasby <[email protected]>
parent: Alena Rybakina <[email protected]>
2 siblings, 0 replies; 77+ messages in thread
From: Jim Nasby @ 2025-03-24 23:02 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Bertrand Drouvot <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
On Fri, Mar 21, 2025 at 2:42 PM Alena Rybakina <[email protected]>
wrote:
> On 13.03.2025 09:42, Bertrand Drouvot wrote:
>
> Hi,
>
> On Wed, Mar 12, 2025 at 05:15:53PM -0500, Jim Nasby wrote:
>
> The usecase I can see here is that we don't want autovac creating so much
> WAL traffic that it starts forcing other backends to have to write WAL out.
> But tracking how many times autovac writes WAL buffers won't help with that
>
> Right, because the one that increments the wal_buffers_full metric could "just"
> be a victim (i.e the one that happens to trigger the WAL buffers disk flush,
> even though other backends contributed most of the buffer usage).
>
>
> (though we also don't want any WAL buffers written by autovac to be counted
> in the system-wide wal_buffers_full:
>
> why? Or do you mean that it would be good to have 2 kinds of metrics: one
> generated by "maintenance" activity and one by "regular" backends?
>
> See below...
> What would be helpful would be a way to determine if autovac was causing
> enough traffic to force other backends to write WAL. Offhand I'm not sure
> how practical that actually is though.
>
> a051e71e28a could help to see how much WAL has by written by the autovac workers.
>
> I still don't think that helps (see below)
> BTW, there's also an argument to be made that autovac should throttle
> itself if we're close to running out of available WAL buffers...
>
> hmm, yeah I think that's an interesting idea OTOH that would mean to "delegate"
> the WAL buffers flush to another backend.
>
> Maybe it does, maybe it doesn't... but now I think you're getting why I'm
complaining about the proposed WAL flush metrics: who *flushes* WAL tells
you absolutely nothing about who generated the WAL. Not only that, but
flushing WAL isn't necessarily even bad: a user backend can't COMMIT
without flushing some amount of WAL (ignoring async-commit of course). That
really casts the whole idea of having stats on who's flushing how much WAL
in a new light: you can NOT use any such metric without a bunch of other
context; including who else was flushing how much WAL, whether WAL had to
absolutely be flushed anyway (ie, at bare minimum a COMMIT must flush
enough WAL to cover the commit record), and even where all the WAL is
coming from in the first place.
Though now that I think about it... if we're reporting how much WAL is
being generated by vacuum, then *maybe* it's helpful to also report how
much WAL is being flushed by vacuum. My emphasis on *maybe* is because it's
fine if autovac is writing more than it flushes, so long as the remainder
is being flushed by the checkpointer and not user backends... but you could
also determine that just by looking at how much WAL backends are flushing.
Basically, I'm leaning towards it would be best to rethink the whole
purpose of reporting WAL flush metrics before we further muddy the waters
by adding vacuum stats about it. At minimum we should have a metric that
shows how much WAL backends flushed because they *had* to due to
synchronous commit settings (which does affect more than just COMMIT).
> I will add it and fix the tests but later and I'll explain why.
>
> I'm working on this issue [0] and try have already created new statistics
> in Statistics Collector to store database and relation vacuum statistics:
> PGSTAT_KIND_VACUUM_DB and PGSTAT_KIND_VACUUM_RELATION.
>
> Vacuum statistics are saved there instead of relation's and database's
> statistic structure, but for some reason it is not possible to find them in
> the hash table when building a snapshot and display them accordingly.
> I have not yet figured out where the error is.
>
> Without solving this problem, committing vacuum statistics is not yet
> possible. An alternative way for us was to refuse some statistics for now
> for relations,
> but we could not agree on which statistics should not be displayed yet and
> for now we are only adding them :).
>
> I understand why this is important to display more vacuum information
> about vacuum statistics - it will allow us to better understand the
> problems of incorrect vacuum settings or, for example, notice a bug in its
> operation.
>
> In order to reduce the memory consumption for storing them for those who
> are not going to use them, I just realized that we need to create a
> separate space for storing the statistics
> I mentioned above (PGSTAT_KIND_VACUUM_DB and PGSTAT_KIND_VACUUM_RELATION),
> there is no other way to do this and I am still trying to complete this
> functionality.
>
> I doubt that I will have time for this by code freeze date and even if I
> do, I will hardly have time for a normal review. There's really a lot more
> to learn related to the stat collector, so
> I'm postponing it to the next commitfest.
>
> Sorry. I'll fix the tests as soon as I finish this part, since they'll
> most likely either break the same way or in some new way.
>
> Tomorrow or the day after tomorrow I will send a diff patch with what I
> have already managed to demonstrate the problem, since I need to bring the
> code to a normal form.
> Maybe someone who worked with the stat collector will suddenly tell me
> where and what I have implemented incorrectly.
>
> --
> Regards,
> Alena Rybakina
> Postgres Professional
>
>
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-03-25 06:12 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
2 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2025-03-25 06:12 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; Jim Nasby <[email protected]>; vignesh C <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>
Hi! I rebased the patches again - PGSTAT_FILE_FORMAT_ID needed to be fixed.
On 21.03.2025 22:42, Alena Rybakina wrote:
>
> I will add it and fix the tests but later and I'll explain why.
>
> I'm working on this issue [0] and try have already created new
> statistics in Statistics Collector to store database and relation
> vacuum statistics: PGSTAT_KIND_VACUUM_DB and PGSTAT_KIND_VACUUM_RELATION.
>
> Vacuum statistics are saved there instead of relation's and database's
> statistic structure, but for some reason it is not possible to find
> them in the hash table when building a snapshot and display them
> accordingly.
> I have not yet figured out where the error is.
>
> Without solving this problem, committing vacuum statistics is not yet
> possible. An alternative way for us was to refuse some statistics for
> now for relations,
> but we could not agree on which statistics should not be displayed yet
> and for now we are only adding them :).
>
> I understand why this is important to display more vacuum information
> about vacuum statistics - it will allow us to better understand the
> problems of incorrect vacuum settings or, for example, notice a bug in
> its operation.
>
> In order to reduce the memory consumption for storing them for those
> who are not going to use them, I just realized that we need to create
> a separate space for storing the statistics
> I mentioned above (PGSTAT_KIND_VACUUM_DB and
> PGSTAT_KIND_VACUUM_RELATION), there is no other way to do this and I
> am still trying to complete this functionality.
>
> I doubt that I will have time for this by code freeze date and even if
> I do, I will hardly have time for a normal review. There's really a
> lot more to learn related to the stat collector, so
> I'm postponing it to the next commitfest.
>
> Sorry. I'll fix the tests as soon as I finish this part, since they'll
> most likely either break the same way or in some new way.
>
> Tomorrow or the day after tomorrow I will send a diff patch with what
> I have already managed to demonstrate the problem, since I need to
> bring the code to a normal form.
> Maybe someone who worked with the stat collector will suddenly tell me
> where and what I have implemented incorrectly.
>
I attached also diff version that contains what I was talking about. The
test case:
create table t (x int);
insert into t select id from generate_series(1,1000) id;
delete from t where id > 900;
vacuum;
select * from pg_stat_vacuum_tables where relname = 't';
--
Regards,
Alena Rybakina
Postgres Professional
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d9b07837831..c620d088204 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -413,7 +413,7 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
} LVRelState;
@@ -525,7 +525,7 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters *counters,
- ExtVacReport *report)
+ PgStat_VacuumRelationCounts *report)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -603,9 +603,9 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
void
extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report)
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report)
{
- memset(report, 0, sizeof(ExtVacReport));
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
extvac_stats_end(rel, &counters->common, report);
report->type = PGSTAT_EXTVAC_INDEX;
@@ -1127,6 +1127,14 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
*
* We are ready to send vacuum statistics information for heap relations.
*/
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime);
+
if(pgstat_track_vacuum_statistics)
{
/* Make generic extended vacuum stats report and
@@ -1135,25 +1143,10 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
- pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime,
- &(vacrel->extVacReport));
+ pgstat_report_tab_vacuum_extstats(vacrel->reloid, true,
+ &(vacrel->extVacReport));
}
- else
- {
- pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime,
- NULL);
- }
pgstat_progress_end_command();
@@ -3349,7 +3342,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3384,9 +3377,8 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
/* Make extended vacuum stats report for index */
extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(vacrel->indoid, true,
+ &extVacReport);
}
/* Revert to the previous phase information for error traceback */
@@ -3414,7 +3406,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3448,9 +3440,8 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
/* Make extended vacuum stats report for index */
extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(vacrel->indoid, true,
+ &extVacReport);
}
/* Revert to the previous phase information for error traceback */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index bd3554c0bfd..75c2e122bf2 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1873,6 +1873,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 739a92bdcc1..e4fa754aab4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2327,6 +2327,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 5fbbcdaabb1..c4b910cd928 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1789,6 +1789,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 000388a565f..c2abed144c4 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,7 +869,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -913,9 +913,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
{
/* Make extended vacuum stats report for index */
extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(RelationGetRelid(indrel), true,
+ &extVacReport);
}
/*
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 9c2443e1ecd..183f7514d2d 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 09fa0fbee57..8a5f355e9bc 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -478,6 +478,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index d5c1e2a2cf5..344f0a24683 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
@@ -449,7 +458,6 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
- memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 5d36d5a2140..db6dba70331 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
@@ -205,50 +203,17 @@ pgstat_drop_relation(Relation rel)
}
}
-/* ---------
- * pgstat_report_vacuum_error() -
- *
- * Tell the collector about an (auto)vacuum interruption.
- * ---------
- */
-void
-pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
-{
- PgStat_EntryRef *entry_ref;
- PgStatShared_Relation *shtabentry;
- PgStat_StatTabEntry *tabentry;
- Oid dboid = MyDatabaseId;
- PgStat_StatDBEntry *dbentry; /* pending database entry */
-
- if (!pgstat_track_counts)
- return;
-
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
- dboid, tableoid, false);
-
- shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
- tabentry = &shtabentry->stats;
-
- tabentry->vacuum_ext.type = m_type;
- pgstat_unlock_entry(entry_ref);
-
- dbentry = pgstat_prep_database_pending(dboid);
- dbentry->vacuum_ext.errors++;
- dbentry->vacuum_ext.type = m_type;
-}
-
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params)
+ TimestampTz starttime)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
- PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -270,8 +235,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
- pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
-
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -307,16 +270,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
-
- if (dboid != InvalidOid)
- {
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dboid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
-
- pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
- pgstat_unlock_entry(entry_ref);
- }
}
/*
@@ -942,6 +895,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
@@ -1044,60 +1003,4 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_updated = trans->updated_pre_truncdrop;
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
-}
-
-static void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info)
-{
- if(!pgstat_track_vacuum_statistics)
- return;
-
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->total_time += src->total_time;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->errors += src->errors;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_TABLE)
- {
- dst->table.pages_scanned += src->table.pages_scanned;
- dst->table.pages_removed += src->table.pages_removed;
- dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
- dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
- dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->table.tuples_frozen += src->table.tuples_frozen;
- dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
- dst->table.index_vacuum_count += src->table.index_vacuum_count;
- dst->table.missed_dead_pages += src->table.missed_dead_pages;
- dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->tuples_deleted += src->tuples_deleted;
- }
- }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c2acdcf0e0e..3605ec98317 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2275,14 +2275,14 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_RelationVacuumPending *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
/* Initialise attributes information in the tuple descriptor */
tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
@@ -2346,18 +2346,16 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
BlessTupleDesc(tupdesc);
- tabentry = pgstat_fetch_stat_tabentry(relid);
+ pending = find_vacuum_relation_entry(relid);
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = &(pending->counts);
i = 0;
@@ -2416,14 +2414,14 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_RelationVacuumPending *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
/* Initialise attributes information in the tuple descriptor */
tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
@@ -2467,18 +2465,16 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
BlessTupleDesc(tupdesc);
- tabentry = pgstat_fetch_stat_tabentry(relid);
+ pending = find_vacuum_relation_entry(relid);
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
- else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+
+ extvacuum = &(pending->counts);
i = 0;
@@ -2523,14 +2519,14 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
Oid dbid = PG_GETARG_OID(0);
- PgStat_StatDBEntry *dbentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumDBCounts *extvacuum;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+
+ PG_RETURN_NULL();
/* Initialise attributes information in the tuple descriptor */
tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
@@ -2570,18 +2566,7 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
BlessTupleDesc(tupdesc);
- dbentry = pgstat_fetch_stat_dbentry(dbid);
-
- if (dbentry == NULL)
- {
- /* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
- extvacuum = &allzero;
- }
- else
- {
- extvacuum = &(dbentry->vacuum_ext);
- }
+ extvacuum = pgstat_prep_vacuum_database_pending(dbid);
i = 0;
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index fb134f3402e..f895151ca09 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -432,5 +432,5 @@ extern double anl_get_next_S(double t, int n, double *stateptr);
extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx *counters);
extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report);
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 66e6e721563..1760b35b5eb 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -119,9 +119,56 @@ typedef enum ExtVacReportType
PGSTAT_EXTVAC_INDEX = 2
} ExtVacReportType;
+/* ----------
+ * PgStat_TableCounts The actual per-table counts kept by a backend
+ *
+ * This struct should contain only actual event counters, because we make use
+ * of pg_memory_is_all_zeros() to detect whether there are any stats updates
+ * to apply.
+ *
+ * It is a component of PgStat_TableStatus (within-backend state).
+ *
+ * Note: for a table, tuples_returned is the number of tuples successfully
+ * fetched by heap_getnext, while tuples_fetched is the number of tuples
+ * successfully fetched by heap_fetch under the control of bitmap indexscans.
+ * For an index, tuples_returned is the number of index entries returned by
+ * the index AM, while tuples_fetched is the number of tuples successfully
+ * fetched by heap_fetch under the control of simple indexscans for this index.
+ *
+ * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
+ * actions, regardless of whether the transaction committed. delta_live_tuples,
+ * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
+ * Note that delta_live_tuples and delta_dead_tuples can be negative!
+ * ----------
+ */
+typedef struct PgStat_TableCounts
+{
+ PgStat_Counter numscans;
+
+ PgStat_Counter tuples_returned;
+ PgStat_Counter tuples_fetched;
+
+ PgStat_Counter tuples_inserted;
+ PgStat_Counter tuples_updated;
+ PgStat_Counter tuples_deleted;
+ PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_newpage_updated;
+ bool truncdropped;
+
+ PgStat_Counter delta_live_tuples;
+ PgStat_Counter delta_dead_tuples;
+ PgStat_Counter changed_tuples;
+
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+} PgStat_TableCounts;
+
/* ----------
*
- * ExtVacReport
+ * PgStat_VacuumRelationCounts
*
* Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
@@ -129,7 +176,7 @@ typedef enum ExtVacReportType
* pages_deleted refer to free space within the index file
* ----------
*/
-typedef struct ExtVacReport
+typedef struct PgStat_VacuumRelationCounts
{
/* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
int64 total_blks_read;
@@ -154,7 +201,6 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
- int32 errors;
int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
ExtVacReportType type; /* heap, index, etc. */
@@ -192,61 +238,44 @@ typedef struct ExtVacReport
int64 pages_deleted; /* number of pages deleted by vacuum */
} index;
} /* per_type_stats */;
-} ExtVacReport;
+} PgStat_VacuumRelationCounts;
-/* ----------
- * PgStat_TableCounts The actual per-table counts kept by a backend
- *
- * This struct should contain only actual event counters, because we make use
- * of pg_memory_is_all_zeros() to detect whether there are any stats updates
- * to apply.
- *
- * It is a component of PgStat_TableStatus (within-backend state).
- *
- * Note: for a table, tuples_returned is the number of tuples successfully
- * fetched by heap_getnext, while tuples_fetched is the number of tuples
- * successfully fetched by heap_fetch under the control of bitmap indexscans.
- * For an index, tuples_returned is the number of index entries returned by
- * the index AM, while tuples_fetched is the number of tuples successfully
- * fetched by heap_fetch under the control of simple indexscans for this index.
- *
- * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
- * actions, regardless of whether the transaction committed. delta_live_tuples,
- * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
- * Note that delta_live_tuples and delta_dead_tuples can be negative!
- * ----------
- */
-typedef struct PgStat_TableCounts
+typedef struct PgStat_VacuumRelationStatus
{
- PgStat_Counter numscans;
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
- PgStat_Counter tuples_returned;
- PgStat_Counter tuples_fetched;
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
- PgStat_Counter tuples_inserted;
- PgStat_Counter tuples_updated;
- PgStat_Counter tuples_deleted;
- PgStat_Counter tuples_hot_updated;
- PgStat_Counter tuples_newpage_updated;
- bool truncdropped;
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
- PgStat_Counter delta_live_tuples;
- PgStat_Counter delta_dead_tuples;
- PgStat_Counter changed_tuples;
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
- PgStat_Counter rev_all_visible_pages;
- PgStat_Counter rev_all_frozen_pages;
+ int64 tuples_deleted; /* tuples deleted by vacuum */
- /*
- * Additional cumulative stat on vacuum operations.
- * Use an expensive structure as an abstraction for different types of
- * relations.
- */
- ExtVacReport vacuum_ext;
-} PgStat_TableCounts;
+ int32 errors;
+ int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+} PgStat_VacuumDBCounts;
/* ----------
* PgStat_TableStatus Per-table status within a backend
@@ -272,6 +301,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -468,8 +503,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
-
- ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -551,8 +584,6 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter rev_all_visible_pages;
PgStat_Counter rev_all_frozen_pages;
-
- ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -760,7 +791,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params);
+ TimestampTz starttime);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -895,6 +926,15 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+pgstat_report_tab_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts *pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts *pgstat_fetch_stat_vacuum_tabentry(Oid relid);
+
/*
* Functions in pgstat_wal.c
*/
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index d5557e6e998..140adbcdbd6 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -439,6 +439,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -607,6 +619,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index f44169fd5a3..454661f9d6a 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
Attachments:
[text/x-patch] v21-0004-Add-documentation-about-the-system-views-that-are-us.patch (24.5K, 3-v21-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 0c0aee02a36c33e68e7a80d31160f6f83cbb6eb0 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 4/4] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 755 +++++++++++++++++++++++++++++++++
1 file changed, 755 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247e..4ad130c6f34 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5074,4 +5074,759 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.34.1
[text/x-patch] v21-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (31.1K, 4-v21-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From a737c545f3d01ab39430deb43a1925cf31cbd863 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 4 Feb 2025 17:57:44 +0300
Subject: [PATCH 3/4] Machinery for grabbing an extended vacuum statistics on
databases.
Database vacuum statistics information is the collected general
vacuum statistics indexes and tables owned by the databases, which
they belong to.
In addition to the fact that there are far fewer databases in a system
than relations, vacuum statistics for a database contain fewer statistics
than relations, but they are enough to indicate that something may be
wrong in the system and prompt the administrator to enable extended
monitoring for relations.
So, buffer, wal, statistics of I/O time of read and writen blocks
statistics will be observed because they are collected for both
tables, indexes. In addition, we show the number of errors caught
during operation of the vacuum only for the error level.
wraparound_failsafe_count is a number of times when the vacuum starts
urgent cleanup to prevent wraparound problem which is critical for
the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 17 ++-
src/backend/catalog/system_views.sql | 27 ++++-
src/backend/utils/activity/pgstat.c | 2 +-
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 46 +++++++-
src/backend/utils/adt/pgstatfuncs.c | 100 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 13 ++-
src/include/pgstat.h | 5 +-
.../vacuum-extending-in-repetable-read.spec | 6 ++
src/test/regress/expected/rules.out | 17 +++
.../expected/vacuum_index_statistics.out | 16 +--
...ut => vacuum_tables_and_db_statistics.out} | 87 +++++++++++++--
src/test/regress/parallel_schedule | 2 +-
.../regress/sql/vacuum_index_statistics.sql | 6 +-
...ql => vacuum_tables_and_db_statistics.sql} | 69 +++++++++++-
16 files changed, 381 insertions(+), 35 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (82%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (81%)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f5aad61afa6..d9b07837831 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -660,7 +660,7 @@ accumulate_heap_vacuum_statistics(LVExtStatCounters *extVacCounters, LVRelState
vacrel->extVacReport.table.missed_dead_tuples += vacrel->missed_dead_tuples;
vacrel->extVacReport.table.missed_dead_pages += vacrel->missed_dead_pages;
vacrel->extVacReport.table.index_vacuum_count += vacrel->num_index_scans;
- vacrel->extVacReport.table.wraparound_failsafe_count += vacrel->wraparound_failsafe_count;
+ vacrel->extVacReport.wraparound_failsafe_count += vacrel->wraparound_failsafe_count;
}
@@ -4065,6 +4065,9 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4080,6 +4083,9 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4095,16 +4101,25 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index c69c60de49b..a895c8789e8 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1472,4 +1472,29 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.errors AS errors
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index e0552f840a0..09fa0fbee57 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-bool pgstat_track_vacuum_statistics = true;
+bool pgstat_track_vacuum_statistics = false;
/* ----------
* state shared with pgstat_*.c
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 05a8ccfdb75..d5c1e2a2cf5 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -449,6 +449,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index cd4ffb50bca..5d36d5a2140 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -205,6 +205,38 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.type = m_type;
+ pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.errors++;
+ dbentry->vacuum_ext.type = m_type;
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
@@ -216,6 +248,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -274,6 +307,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
}
/*
@@ -1021,6 +1064,8 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->blk_write_time += src->blk_write_time;
dst->delay_time += src->delay_time;
dst->total_time += src->total_time;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->errors += src->errors;
if (!accumulate_reltype_specific_info)
return;
@@ -1048,7 +1093,6 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->table.index_vacuum_count += src->table.index_vacuum_count;
dst->table.missed_dead_pages += src->table.missed_dead_pages;
dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
}
else if (dst->type == PGSTAT_EXTVAC_INDEX)
{
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 15fa3de0871..c2acdcf0e0e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2383,7 +2383,7 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
@@ -2513,6 +2513,104 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 191894aacab..259067a4030 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1502,7 +1502,7 @@ struct config_bool ConfigureNamesBool[] =
NULL
},
&pgstat_track_vacuum_statistics,
- true,
+ false,
NULL, NULL, NULL
},
{
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 369806befb6..474e0f86b73 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12498,12 +12498,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,wraparound_failsafe,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 30e7e5537f5..66e6e721563 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -154,6 +154,9 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
+ int32 errors;
+ int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+
ExtVacReportType type; /* heap, index, etc. */
/* ----------
@@ -183,7 +186,6 @@ typedef struct ExtVacReport
int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} table;
struct
{
@@ -762,6 +764,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5893d89573d..cfec3159580 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -18,6 +18,9 @@ teardown
}
session s1
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s1_begin_repeatable_read {
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
select count(ival) from test_vacuum_stat_isolation where id>900;
@@ -25,6 +28,9 @@ step s1_begin_repeatable_read {
step s1_commit { COMMIT; }
session s2
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 640f799b7c1..c5df58ebd72 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2262,6 +2262,23 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schemaname,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index e00a0fc683c..9e5d33342c9 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -16,8 +16,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -33,12 +37,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -181,3 +180,4 @@ WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
(1 row)
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 82%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index b5ea9c9ab1e..0300e7b6276 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,7 +6,6 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
--------------
@@ -16,8 +15,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -37,12 +40,12 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -225,3 +228,69 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
(1 row)
DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 3498ce06635..37fa1c67a49 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index ae146e1d23f..9b7e645187d 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -14,8 +14,7 @@ SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -33,7 +32,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -149,3 +148,4 @@ FROM pg_stat_vacuum_indexes vt, pg_class c
WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 81%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 5bc34bec64b..ca7dbde9387 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,15 +7,13 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -36,7 +34,13 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -180,4 +184,59 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+DROP TABLE vestat CASCADE;
+
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
\ No newline at end of file
--
2.34.1
[text/x-patch] v21-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (58.0K, 5-v21-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 0abab1585a59bb8ed862022093bb898fbbbb8790 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 27 Feb 2025 20:42:05 +0300
Subject: [PATCH 2/4] Machinery for grabbing an extended vacuum statistics on
index relations.
They are gathered separatelly from table statistics.
As for tables, we gather vacuum shared buffers statistics for index relations like
value of total_blks_hit, total_blks_read, total_blks_dirtied, wal statistics, io time
during flushing buffer pages to disk, delay and total time.
Due to the fact that such statistics are common as for tables, as for indexes we
set them in the union ExtVacReport structure. We only added some determination 'type'
field to highlight what kind belong to these statistics: PGSTAT_EXTVAC_TABLE or
PGSTAT_EXTVAC_INDEX. Generally, PGSTAT_EXTVAC_INVALID type leads to wrong code process.
Some statistics belong only one type of both tables or indexes. So, we added substructures
sych table and index inside ExtVacReport structure.
Therefore, we gather only for tables such statistics like number of scanned, removed pages,
their charecteristics according VM (all-visible and frozen). In addition, for tables we
gather number frozen, deleted and recently dead tuples and how many times vacuum processed
indexes for tables.
Controversally for indexes we gather number of deleted pages and deleted tuples only.
As for tables, deleted pages and deleted tuples reflect the overall performance of the vacuum
for the index relationship.
Since the vacuum cleans up references to tuple indexes before cleaning up table tuples,
which adds some complexity to the vacuum process, namely the vacuum switches from cleaning up
a table to its indexes and back during its operation, we need to save the vacuum statistics
collected for the heap before it starts cleaning up the indexes.
That's why it's necessary to track the vacuum statistics for the heap several times during
the vacuum procedure. To avoid sending the statistics to the Cumulative Statistics System
several times, we save these statistics in the LVRelState structure and only after vacuum
finishes cleaning up the heap, it sends them to the Cumulative Statistics System.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 292 ++++++++++++++----
src/backend/catalog/system_views.sql | 32 ++
src/backend/commands/vacuumparallel.c | 14 +
src/backend/utils/activity/pgstat.c | 4 +
src/backend/utils/activity/pgstat_relation.c | 48 ++-
src/backend/utils/adt/pgstatfuncs.c | 133 +++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 25 ++
src/include/pgstat.h | 58 +++-
.../vacuum-extending-in-repetable-read.out | 4 +-
src/test/regress/expected/rules.out | 22 ++
.../expected/vacuum_index_statistics.out | 183 +++++++++++
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 151 +++++++++
15 files changed, 873 insertions(+), 105 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 592b018c300..f5aad61afa6 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -291,6 +291,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -411,6 +412,8 @@ typedef struct LVRelState
BlockNumber eager_scan_remaining_fails;
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReport extVacReport;
} LVRelState;
@@ -422,19 +425,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-/*
- * Counters and usage data for extended stats tracking.
- */
-typedef struct LVExtStatCounters
-{
- TimestampTz starttime;
- WalUsage walusage;
- BufferUsage bufusage;
- double VacuumDelayTime;
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-} LVExtStatCounters;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static void heap_vacuum_eager_scan_setup(LVRelState *vacrel,
@@ -556,27 +546,25 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
endtime = GetCurrentTimestamp();
TimestampDifference(counters->starttime, endtime, &secs, &usecs);
- memset(report, 0, sizeof(ExtVacReport));
-
/*
* Fill additional statistics on a vacuum processing operation.
*/
- report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
- report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
- report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
- report->total_blks_written = bufusage.shared_blks_written;
+ report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written += bufusage.shared_blks_written;
- report->wal_records = walusage.wal_records;
- report->wal_fpi = walusage.wal_fpi;
- report->wal_bytes = walusage.wal_bytes;
+ report->wal_records += walusage.wal_records;
+ report->wal_fpi += walusage.wal_fpi;
+ report->wal_bytes += walusage.wal_bytes;
- report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
- report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
- report->total_time = secs * 1000. + usecs / 1000.;
+ report->total_time += secs * 1000. + usecs / 1000.;
if (!rel->pgstat_info || !pgstat_track_counts)
/*
@@ -585,12 +573,96 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
*/
return;
- report->blks_fetched =
+ report->blks_fetched +=
rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
- report->blks_hit =
+ report->blks_hit +=
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Set initial values for common heap and index statistics*/
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ memset(report, 0, sizeof(ExtVacReport));
+
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVExtStatCounters *extVacCounters, LVRelState *vacrel)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReport.type = PGSTAT_EXTVAC_TABLE;
+ vacrel->extVacReport.table.pages_scanned += vacrel->scanned_pages;
+ vacrel->extVacReport.table.pages_removed += vacrel->removed_pages;
+ vacrel->extVacReport.table.vm_new_frozen_pages += vacrel->vm_new_frozen_pages;
+ vacrel->extVacReport.table.vm_new_visible_pages += vacrel->vm_new_visible_pages;
+ vacrel->extVacReport.table.vm_new_visible_frozen_pages += vacrel->vm_new_visible_frozen_pages;
+ vacrel->extVacReport.tuples_deleted += vacrel->tuples_deleted;
+ vacrel->extVacReport.table.tuples_frozen += vacrel->tuples_frozen;
+ vacrel->extVacReport.table.recently_dead_tuples += vacrel->recently_dead_tuples;
+ vacrel->extVacReport.table.recently_dead_tuples += vacrel->recently_dead_tuples;
+ vacrel->extVacReport.table.missed_dead_tuples += vacrel->missed_dead_tuples;
+ vacrel->extVacReport.table.missed_dead_pages += vacrel->missed_dead_pages;
+ vacrel->extVacReport.table.index_vacuum_count += vacrel->num_index_scans;
+ vacrel->extVacReport.table.wraparound_failsafe_count += vacrel->wraparound_failsafe_count;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -748,13 +820,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
LVExtStatCounters extVacCounters;
- ExtVacReport extVacReport;
char **indnames = NULL;
- ExtVacReport allzero;
-
- /* Initialize vacuum statistics */
- memset(&allzero, 0, sizeof(ExtVacReport));
- extVacReport = allzero;
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -774,7 +840,6 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
- extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -961,6 +1026,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
*/
lazy_scan_heap(vacrel);
+ extvac_stats_start(rel, &extVacCounters);
+
/*
* Free resources managed by dead_items_alloc. This ends parallel mode in
* passing when necessary.
@@ -1048,26 +1115,6 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
- /* Make generic extended vacuum stats report */
- extvac_stats_end(rel, &extVacCounters, &extVacReport);
-
- if(pgstat_track_vacuum_statistics)
- {
- /* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
- extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
- extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
- extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
- extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- }
-
/*
* Report results to the cumulative stats system, too.
*
@@ -1077,14 +1124,37 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
* It seems like a good idea to err on the side of not vacuuming again too
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
+ *
+ * We are ready to send vacuum statistics information for heap relations.
*/
- pgstat_report_vacuum(RelationGetRelid(rel),
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
+ vacrel->missed_dead_tuples,
starttime,
- &extVacReport);
+ &(vacrel->extVacReport));
+
+ }
+ else
+ {
+ pgstat_report_vacuum(RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+ }
+
pgstat_progress_end_command();
if (instrument)
@@ -1356,6 +1426,7 @@ lazy_scan_heap(LVRelState *vacrel)
PROGRESS_VACUUM_MAX_DEAD_TUPLE_BYTES
};
int64 initprog_val[3];
+ LVExtStatCounters extVacCounters;
/* Report that we're scanning the heap, advertising total # of blocks */
initprog_val[0] = PROGRESS_VACUUM_PHASE_SCAN_HEAP;
@@ -1370,6 +1441,13 @@ lazy_scan_heap(LVRelState *vacrel)
vacrel->next_unskippable_eager_scanned = false;
vacrel->next_unskippable_vmbuffer = InvalidBuffer;
+ /*
+ * Due to the fact that vacuum heap processing needs their index vacuuming
+ * we need to track them separately and accumulate heap vacuum statistics
+ * separately. So last processes are related to only heap vacuuming process.
+ */
+ extvac_stats_start(vacrel->rel, &extVacCounters);
+
/* Set up the read stream for vacuum's first pass through the heap */
stream = read_stream_begin_relation(READ_STREAM_MAINTENANCE,
vacrel->bstrategy,
@@ -1429,8 +1507,26 @@ lazy_scan_heap(LVRelState *vacrel)
/* Perform a round of index and heap vacuuming */
vacrel->consider_bypass_optimization = false;
+
+ /*
+ * Lazy vacuum stage includes index vacuuming and cleaning up stage, so
+ * we prefer tracking them separately.
+ * Before starting to process the indexes save the current heap statistics
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
+
lazy_vacuum(vacrel);
+ /*
+ * After completion lazy vacuum, we start again tracking vacuum statistics for
+ * heap-related objects like FSM, VM, provide heap prunning.
+ * It seems dangerously that we have start tracking but there are no end, but
+ * it is safe. The end tracking is located before lazy vacuum stage in the same
+ * loop or after it.
+ */
+ extvac_stats_start(vacrel->rel, &extVacCounters);
+
/*
* Vacuum the Free Space Map to make newly-freed space visible on
* upper-level FSM pages. Note that blkno is the previously
@@ -1653,6 +1749,12 @@ lazy_scan_heap(LVRelState *vacrel)
read_stream_end(stream);
+ /*
+ * Vacuum can process lazy vacuum again and we save heap statistics now
+ * just in case in tend to avoid collecting vacuum index statistics again.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
/*
* Do index vacuuming (call each index's ambulkdelete routine), then do
* related heap vacuuming
@@ -1660,6 +1762,12 @@ lazy_scan_heap(LVRelState *vacrel)
if (vacrel->dead_items_info->num_items > 0)
lazy_vacuum(vacrel);
+ /*
+ * We need to take into account heap vacuum statistics during process of
+ * FSM.
+ */
+ extvac_stats_start(vacrel->rel, &extVacCounters);
+
/*
* Vacuum the remainder of the Free Space Map. We must do this whether or
* not there were indexes, and whether or not we bypassed index vacuuming.
@@ -1672,6 +1780,10 @@ lazy_scan_heap(LVRelState *vacrel)
/* report all blocks vacuumed */
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, rel_pages);
+ /* Before starting final index clan up stage save heap statistics */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
+
/* Do final index cleanup (call each index's amvacuumcleanup routine) */
if (vacrel->nindexes > 0 && vacrel->do_index_cleanup)
lazy_cleanup_all_indexes(vacrel);
@@ -2589,6 +2701,7 @@ static void
lazy_vacuum(LVRelState *vacrel)
{
bool bypass;
+ LVExtStatCounters extVacCounters;
/* Should not end up here with no indexes */
Assert(vacrel->nindexes > 0);
@@ -2601,6 +2714,9 @@ lazy_vacuum(LVRelState *vacrel)
return;
}
+ /* Set initial statistics values to gather vacuum statistics for the heap */
+ extvac_stats_start(vacrel->rel, &extVacCounters);
+
/*
* Consider bypassing index vacuuming (and heap vacuuming) entirely.
*
@@ -2657,6 +2773,14 @@ lazy_vacuum(LVRelState *vacrel)
TidStoreMemoryUsage(vacrel->dead_items) < 32 * 1024 * 1024);
}
+ /*
+ * Vacuum is likely to vacuum indexes again, so save vacuum statistics for
+ * heap relations now.
+ * The vacuum process below doesn't contain any useful statistics information
+ * for heap if indexes won't be processed, but we will track them separately.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+
if (bypass)
{
/*
@@ -2673,11 +2797,21 @@ lazy_vacuum(LVRelState *vacrel)
}
else if (lazy_vacuum_all_indexes(vacrel))
{
- /*
- * We successfully completed a round of index vacuuming. Do related
- * heap vacuuming now.
- */
- lazy_vacuum_heap_rel(vacrel);
+ /* Now the vacuum is going to process heap relation, so
+ * we need to set intial statistic values for tracking.
+ */
+
+ /* Set initial statistics values to gather vacuum statistics for the heap */
+ extvac_stats_start(vacrel->rel, &extVacCounters);
+
+ /*
+ * We successfully completed a round of index vacuuming. Do related
+ * heap vacuuming now.
+ */
+ lazy_vacuum_heap_rel(vacrel);
+
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
}
else
{
@@ -3214,6 +3348,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3232,6 +3371,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3240,6 +3380,15 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3264,6 +3413,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3283,12 +3437,22 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 603bf97e042..c69c60de49b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1441,3 +1441,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 7924c526cb0..000388a565f 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,15 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 521a802fe2d..e0552f840a0 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1179,6 +1179,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 0272dd1f393..cd4ffb50bca 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -1007,6 +1007,9 @@ static void
pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
bool accumulate_reltype_specific_info)
{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
dst->total_blks_read += src->total_blks_read;
dst->total_blks_hit += src->total_blks_hit;
dst->total_blks_dirtied += src->total_blks_dirtied;
@@ -1022,20 +1025,35 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
- dst->vm_new_visible_pages += src->vm_new_visible_pages;
- dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->missed_dead_pages += src->missed_dead_pages;
- dst->missed_dead_tuples += src->missed_dead_tuples;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->tuples_deleted += src->tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 416f3c51b0c..15fa3de0871 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2372,18 +2372,19 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2402,6 +2403,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 40767e44601..191894aacab 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1498,7 +1498,7 @@ struct config_bool ConfigureNamesBool[] =
},
{
{"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
- gettext_noop("Collects vacuum statistics for table relations."),
+ gettext_noop("Collects vacuum statistics for relations."),
NULL
},
&pgstat_track_vacuum_statistics,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6bac3cbc3eb..369806befb6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12497,4 +12497,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 6d1b2991ce5..fb134f3402e 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -295,6 +296,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -408,4 +429,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 3522223d9ca..30e7e5537f5 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,11 +111,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -144,18 +152,44 @@ typedef struct ExtVacReport
double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
double total_time; /* total time of a vacuum operation, in msec */
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
- int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
- int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
- int64 missed_dead_pages; /* pages with missed dead tuples */
int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
- int64 index_vacuum_count; /* the number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 87f7e40b4a6..6d960423912 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -34,7 +34,7 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
(1 row)
step s1_commit: COMMIT;
@@ -48,6 +48,6 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
(1 row)
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f4bba9cc30c..640f799b7c1 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2262,6 +2262,28 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schemaname,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
rel.relname,
stats.relid,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..e00a0fc683c
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+ relid | schemaname | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+------------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+(0 rows)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 99bcc46efcc..3498ce06635 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..ae146e1d23f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,151 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] v21-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (71.1K, 6-v21-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 5973c92e91f1a409067a864e6964dfa6e922cb8b Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 27 Feb 2025 20:01:38 +0300
Subject: [PATCH 1/4] Machinery for grabbing an extended vacuum statistics on
table relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen is a number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible is a number of pages that are marked as all-visible in vm during
vacuum.
wraparound_failsafe_count is a number of times when the vacuum starts urgent cleanup
to prevent wraparound problem which is critical for the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 150 +++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 52 +++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 46 +++-
src/backend/utils/adt/pgstatfuncs.c | 147 ++++++++++++
src/backend/utils/error/elog.c | 13 +
src/backend/utils/misc/guc_tables.c | 9 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 80 +++++-
src/include/utils/elog.h | 1 +
.../vacuum-extending-in-repetable-read.out | 53 ++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 53 ++++
src/test/regress/expected/rules.out | 44 +++-
.../expected/vacuum_tables_statistics.out | 227 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 183 ++++++++++++++
22 files changed, 1095 insertions(+), 16 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 2cbcf5e5db2..592b018c300 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -290,6 +290,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -408,6 +409,8 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} LVRelState;
@@ -419,6 +422,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -475,6 +490,106 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
@@ -632,7 +747,14 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -652,7 +774,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -669,6 +791,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -758,6 +881,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->vm_new_visible_frozen_pages = 0;
vacrel->vm_new_frozen_pages = 0;
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
+ vacrel->wraparound_failsafe_count = 0;
/*
* Get cutoffs that determine which deleted tuples are considered DEAD,
@@ -924,6 +1048,26 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ }
+
/*
* Report results to the cumulative stats system, too.
*
@@ -939,7 +1083,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime);
+ starttime,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -2957,6 +3102,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[2] = {0, 0};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count ++;
/*
* Abandon use of a buffer access strategy to allow use of all of
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 745a04ef26e..07623a045fa 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 31d269b7ee0..603bf97e042 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -700,7 +700,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_vacuum_time(C.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1391,3 +1393,51 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.vm_new_frozen_pages AS vm_new_frozen_pages,
+ stats.vm_new_visible_pages AS vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ stats.missed_dead_pages AS missed_dead_pages,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.recently_dead_tuples AS recently_dead_tuples,
+ stats.missed_dead_tuples AS missed_dead_tuples,
+
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index f0a7b87808d..1ef67299d48 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -115,6 +115,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2512,6 +2515,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 2b9d548cdeb..7924c526cb0 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1054,6 +1054,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 40063085073..521a802fe2d 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = true;
/* ----------
* state shared with pgstat_*.c
@@ -260,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -900,7 +899,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -969,7 +967,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1085,7 +1083,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1136,7 +1134,7 @@ pgstat_prep_snapshot(void)
}
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index d64595a165c..0272dd1f393 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -209,7 +211,7 @@ pgstat_drop_relation(Relation rel)
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime)
+ TimestampTz starttime, ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -235,6 +237,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -872,6 +876,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -995,3 +1002,40 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
+ dst->vm_new_visible_pages += src->vm_new_visible_pages;
+ dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->missed_dead_pages += src->missed_dead_pages;
+ dst->missed_dead_tuples += src->missed_dead_tuples;
+
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 97af7c6554f..416f3c51b0c 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2258,3 +2264,144 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 860bbd40d42..4da8d3f87fd 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 989825d3a9c..40767e44601 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1496,6 +1496,15 @@ struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
+ gettext_noop("Collects vacuum statistics for table relations."),
+ NULL
+ },
+ &pgstat_track_vacuum_statistics,
+ true,
+ NULL, NULL, NULL
+ },
{
{"track_wal_io_timing", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects timing statistics for WAL I/O activity."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 0b9e3066bde..25a8f931983 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -658,6 +658,7 @@
#track_wal_io_timing = off
#track_functions = none # none, pl, all
#stats_fetch_consistency = cache # cache, none, snapshot
+#track_vacuum_statistics = off
# - Monitoring -
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0d29ef50ff2..6bac3cbc3eb 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12479,4 +12479,22 @@
proargtypes => 'int4',
prosrc => 'gist_stratnum_common' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int4,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index bc37a80dc74..6d1b2991ce5 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -327,6 +327,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 5bfe19e66be..3522223d9ca 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,6 +111,53 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* the number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -153,6 +200,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -211,7 +268,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB7
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB8
typedef struct PgStat_ArchiverStats
{
@@ -375,6 +432,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -453,6 +512,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autovacuum_time;
PgStat_Counter total_analyze_time;
PgStat_Counter total_autoanalyze_time;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -660,7 +724,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime);
+ TimestampTz starttime, ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -711,6 +775,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -799,6 +874,7 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
/*
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 855c147325b..b403ffcc090 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..87f7e40b4a6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..e93dd4f626c 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5893d89573d
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,53 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 47478969135..f4bba9cc30c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1808,7 +1808,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_vacuum_time(c.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2201,7 +2203,9 @@ pg_stat_sys_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2253,9 +2257,43 @@ pg_stat_user_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.missed_dead_pages,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.missed_dead_tuples,
+ stats.wraparound_failsafe,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..b5ea9c9ab1e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,227 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+ relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_scanned | pages_removed | vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | missed_dead_pages | tuples_deleted | tuples_frozen | recently_dead_tuples | missed_dead_tuples | index_vacuum_count | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+---------------+---------------------+----------------------+-----------------------------+-------------------+----------------+---------------+----------------------+--------------------+--------------------+-------------+---------+-----------+---------------+----------------+------------+------------
+ vestat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
+(1 row)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
+---------------------+----------------------+----------------------+-----------------------+-----------------------------
+ 0 | 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | f | f
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 0a35f2f8f6a..99bcc46efcc 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..5bc34bec64b
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/plain] vacuum_moved_new_entry.diff.no-cfbot (29.1K, 7-vacuum_moved_new_entry.diff.no-cfbot)
download | inline diff:
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d9b07837831..c620d088204 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -413,7 +413,7 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
} LVRelState;
@@ -525,7 +525,7 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters *counters,
- ExtVacReport *report)
+ PgStat_VacuumRelationCounts *report)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -603,9 +603,9 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
void
extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report)
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report)
{
- memset(report, 0, sizeof(ExtVacReport));
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
extvac_stats_end(rel, &counters->common, report);
report->type = PGSTAT_EXTVAC_INDEX;
@@ -1127,6 +1127,14 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
*
* We are ready to send vacuum statistics information for heap relations.
*/
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime);
+
if(pgstat_track_vacuum_statistics)
{
/* Make generic extended vacuum stats report and
@@ -1135,25 +1143,10 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
- pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime,
- &(vacrel->extVacReport));
+ pgstat_report_tab_vacuum_extstats(vacrel->reloid, true,
+ &(vacrel->extVacReport));
}
- else
- {
- pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime,
- NULL);
- }
pgstat_progress_end_command();
@@ -3349,7 +3342,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3384,9 +3377,8 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
/* Make extended vacuum stats report for index */
extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(vacrel->indoid, true,
+ &extVacReport);
}
/* Revert to the previous phase information for error traceback */
@@ -3414,7 +3406,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3448,9 +3440,8 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
/* Make extended vacuum stats report for index */
extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(vacrel->indoid, true,
+ &extVacReport);
}
/* Revert to the previous phase information for error traceback */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index bd3554c0bfd..75c2e122bf2 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1873,6 +1873,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 739a92bdcc1..e4fa754aab4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2327,6 +2327,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 5fbbcdaabb1..c4b910cd928 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1789,6 +1789,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 000388a565f..c2abed144c4 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,7 +869,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -913,9 +913,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
{
/* Make extended vacuum stats report for index */
extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(RelationGetRelid(indrel), true,
+ &extVacReport);
}
/*
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 9c2443e1ecd..183f7514d2d 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 09fa0fbee57..8a5f355e9bc 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -478,6 +478,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index d5c1e2a2cf5..344f0a24683 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
@@ -449,7 +458,6 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
- memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 5d36d5a2140..db6dba70331 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
@@ -205,50 +203,17 @@ pgstat_drop_relation(Relation rel)
}
}
-/* ---------
- * pgstat_report_vacuum_error() -
- *
- * Tell the collector about an (auto)vacuum interruption.
- * ---------
- */
-void
-pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
-{
- PgStat_EntryRef *entry_ref;
- PgStatShared_Relation *shtabentry;
- PgStat_StatTabEntry *tabentry;
- Oid dboid = MyDatabaseId;
- PgStat_StatDBEntry *dbentry; /* pending database entry */
-
- if (!pgstat_track_counts)
- return;
-
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
- dboid, tableoid, false);
-
- shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
- tabentry = &shtabentry->stats;
-
- tabentry->vacuum_ext.type = m_type;
- pgstat_unlock_entry(entry_ref);
-
- dbentry = pgstat_prep_database_pending(dboid);
- dbentry->vacuum_ext.errors++;
- dbentry->vacuum_ext.type = m_type;
-}
-
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params)
+ TimestampTz starttime)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
- PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -270,8 +235,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
- pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
-
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -307,16 +270,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
-
- if (dboid != InvalidOid)
- {
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dboid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
-
- pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
- pgstat_unlock_entry(entry_ref);
- }
}
/*
@@ -942,6 +895,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
@@ -1044,60 +1003,4 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_updated = trans->updated_pre_truncdrop;
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
-}
-
-static void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info)
-{
- if(!pgstat_track_vacuum_statistics)
- return;
-
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->total_time += src->total_time;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->errors += src->errors;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_TABLE)
- {
- dst->table.pages_scanned += src->table.pages_scanned;
- dst->table.pages_removed += src->table.pages_removed;
- dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
- dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
- dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->table.tuples_frozen += src->table.tuples_frozen;
- dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
- dst->table.index_vacuum_count += src->table.index_vacuum_count;
- dst->table.missed_dead_pages += src->table.missed_dead_pages;
- dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->tuples_deleted += src->tuples_deleted;
- }
- }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c2acdcf0e0e..3605ec98317 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2275,14 +2275,14 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_RelationVacuumPending *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
/* Initialise attributes information in the tuple descriptor */
tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
@@ -2346,18 +2346,16 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
BlessTupleDesc(tupdesc);
- tabentry = pgstat_fetch_stat_tabentry(relid);
+ pending = find_vacuum_relation_entry(relid);
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = &(pending->counts);
i = 0;
@@ -2416,14 +2414,14 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_RelationVacuumPending *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
/* Initialise attributes information in the tuple descriptor */
tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
@@ -2467,18 +2465,16 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
BlessTupleDesc(tupdesc);
- tabentry = pgstat_fetch_stat_tabentry(relid);
+ pending = find_vacuum_relation_entry(relid);
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
- else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+
+ extvacuum = &(pending->counts);
i = 0;
@@ -2523,14 +2519,14 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
Oid dbid = PG_GETARG_OID(0);
- PgStat_StatDBEntry *dbentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumDBCounts *extvacuum;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+
+ PG_RETURN_NULL();
/* Initialise attributes information in the tuple descriptor */
tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
@@ -2570,18 +2566,7 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
BlessTupleDesc(tupdesc);
- dbentry = pgstat_fetch_stat_dbentry(dbid);
-
- if (dbentry == NULL)
- {
- /* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
- extvacuum = &allzero;
- }
- else
- {
- extvacuum = &(dbentry->vacuum_ext);
- }
+ extvacuum = pgstat_prep_vacuum_database_pending(dbid);
i = 0;
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index fb134f3402e..f895151ca09 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -432,5 +432,5 @@ extern double anl_get_next_S(double t, int n, double *stateptr);
extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx *counters);
extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report);
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 66e6e721563..1760b35b5eb 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -119,9 +119,56 @@ typedef enum ExtVacReportType
PGSTAT_EXTVAC_INDEX = 2
} ExtVacReportType;
+/* ----------
+ * PgStat_TableCounts The actual per-table counts kept by a backend
+ *
+ * This struct should contain only actual event counters, because we make use
+ * of pg_memory_is_all_zeros() to detect whether there are any stats updates
+ * to apply.
+ *
+ * It is a component of PgStat_TableStatus (within-backend state).
+ *
+ * Note: for a table, tuples_returned is the number of tuples successfully
+ * fetched by heap_getnext, while tuples_fetched is the number of tuples
+ * successfully fetched by heap_fetch under the control of bitmap indexscans.
+ * For an index, tuples_returned is the number of index entries returned by
+ * the index AM, while tuples_fetched is the number of tuples successfully
+ * fetched by heap_fetch under the control of simple indexscans for this index.
+ *
+ * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
+ * actions, regardless of whether the transaction committed. delta_live_tuples,
+ * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
+ * Note that delta_live_tuples and delta_dead_tuples can be negative!
+ * ----------
+ */
+typedef struct PgStat_TableCounts
+{
+ PgStat_Counter numscans;
+
+ PgStat_Counter tuples_returned;
+ PgStat_Counter tuples_fetched;
+
+ PgStat_Counter tuples_inserted;
+ PgStat_Counter tuples_updated;
+ PgStat_Counter tuples_deleted;
+ PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_newpage_updated;
+ bool truncdropped;
+
+ PgStat_Counter delta_live_tuples;
+ PgStat_Counter delta_dead_tuples;
+ PgStat_Counter changed_tuples;
+
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+} PgStat_TableCounts;
+
/* ----------
*
- * ExtVacReport
+ * PgStat_VacuumRelationCounts
*
* Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
@@ -129,7 +176,7 @@ typedef enum ExtVacReportType
* pages_deleted refer to free space within the index file
* ----------
*/
-typedef struct ExtVacReport
+typedef struct PgStat_VacuumRelationCounts
{
/* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
int64 total_blks_read;
@@ -154,7 +201,6 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
- int32 errors;
int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
ExtVacReportType type; /* heap, index, etc. */
@@ -192,61 +238,44 @@ typedef struct ExtVacReport
int64 pages_deleted; /* number of pages deleted by vacuum */
} index;
} /* per_type_stats */;
-} ExtVacReport;
+} PgStat_VacuumRelationCounts;
-/* ----------
- * PgStat_TableCounts The actual per-table counts kept by a backend
- *
- * This struct should contain only actual event counters, because we make use
- * of pg_memory_is_all_zeros() to detect whether there are any stats updates
- * to apply.
- *
- * It is a component of PgStat_TableStatus (within-backend state).
- *
- * Note: for a table, tuples_returned is the number of tuples successfully
- * fetched by heap_getnext, while tuples_fetched is the number of tuples
- * successfully fetched by heap_fetch under the control of bitmap indexscans.
- * For an index, tuples_returned is the number of index entries returned by
- * the index AM, while tuples_fetched is the number of tuples successfully
- * fetched by heap_fetch under the control of simple indexscans for this index.
- *
- * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
- * actions, regardless of whether the transaction committed. delta_live_tuples,
- * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
- * Note that delta_live_tuples and delta_dead_tuples can be negative!
- * ----------
- */
-typedef struct PgStat_TableCounts
+typedef struct PgStat_VacuumRelationStatus
{
- PgStat_Counter numscans;
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
- PgStat_Counter tuples_returned;
- PgStat_Counter tuples_fetched;
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
- PgStat_Counter tuples_inserted;
- PgStat_Counter tuples_updated;
- PgStat_Counter tuples_deleted;
- PgStat_Counter tuples_hot_updated;
- PgStat_Counter tuples_newpage_updated;
- bool truncdropped;
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
- PgStat_Counter delta_live_tuples;
- PgStat_Counter delta_dead_tuples;
- PgStat_Counter changed_tuples;
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
- PgStat_Counter rev_all_visible_pages;
- PgStat_Counter rev_all_frozen_pages;
+ int64 tuples_deleted; /* tuples deleted by vacuum */
- /*
- * Additional cumulative stat on vacuum operations.
- * Use an expensive structure as an abstraction for different types of
- * relations.
- */
- ExtVacReport vacuum_ext;
-} PgStat_TableCounts;
+ int32 errors;
+ int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+} PgStat_VacuumDBCounts;
/* ----------
* PgStat_TableStatus Per-table status within a backend
@@ -272,6 +301,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -468,8 +503,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
-
- ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -551,8 +584,6 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter rev_all_visible_pages;
PgStat_Counter rev_all_frozen_pages;
-
- ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -760,7 +791,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params);
+ TimestampTz starttime);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -895,6 +926,15 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+pgstat_report_tab_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts *pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts *pgstat_fetch_stat_vacuum_tabentry(Oid relid);
+
/*
* Functions in pgstat_wal.c
*/
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index d5557e6e998..140adbcdbd6 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -439,6 +439,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -607,6 +619,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index f44169fd5a3..454661f9d6a 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-04-22 18:23 Andrei Lepikhov <[email protected]>
parent: Alexander Korotkov <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Andrei Lepikhov @ 2025-04-22 18:23 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; Alena Rybakina <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; jian he <[email protected]>
On 10/28/24 14:40, Alexander Korotkov wrote:
> On Sun, Aug 25, 2024 at 6:59 PM Alena Rybakina
>> If I missed something or misunderstood, can you explain in more detail?
>
> Actually, I mean why do we need a possibility to return statistics for
> all tables/indexes in one function call? User anyway is supposed to
> use pg_stat_vacuum_indexes/pg_stat_vacuum_tables view, which do
> function calls one per relation. I suppose we can get rid of
> possibility to get all the objects in one function call and just
> return a tuple from the functions like other pgstatfuncs.c functions
> do.
I suppose it was designed this way because databases may contain
thousands of tables and indexes - remember, at least, partitions. But it
may be okay to use the SRF_FIRSTCALL_INIT / SRF_RETURN_NEXT API. I think
by registering a prosupport routine predicting cost and rows of these
calls, we may let the planner build adequate plans for queries involving
those stats - people will definitely join it with something else in the
database.
--
regards, Andrei Lepikhov
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-05-09 12:03 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2025-05-09 12:03 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; pgsql-hackers; +Cc: Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
Hi, all!
On 25.03.2025 09:12, Alena Rybakina wrote:
>
> Hi! I rebased the patches again - PGSTAT_FILE_FORMAT_ID needed to be
> fixed.
>
On 05.02.2025 09:59, Alexander Korotkov wrote:
> What is the point for disabling pgstat_track_vacuum_statistics then?
> I don't see it saves any valuable resources. The original point by
> Masahiko Sawada was growth of data structures in times [1] (and
> corresponding memory consumption especially with large number of
> tables). Now, disabling pgstat_track_vacuum_statistics only saves
> some cycles of pgstat_accumulate_extvac_stats(), and that seems
> insignificant.
>
> I see that we use hash tables with static element size. So, we can't
> save space by dynamically changing entries size on the base of GUC.
> But could we move vacuum statistics to separate hash tables? When GUC
> is disabled, new hash tables could be just empty.
>
> Links
> 1.https://www.postgresql.org/message-id/CAD21AoD66b3u28n%3D73kudgMp5wiGiyYUN9LuC9z2ka6YTru5Gw%40mail.g...
I did a rebase and finished the part with storing statistics separately
from the relation statistics - now it is possible to disable the
collection of statistics for relationsh using gucs and
this allows us to solve the problem with the memory consumed.
For now I have formatted all this as a diff file. The diff file must be
applied after all patches have been applied. While looking at it I
noticed that the code requires significant code refactoring, so I will
do that.
--
Regards,
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] v22-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (71.3K, 3-v22-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 0990e725306d9488291d022a37bc17eb66d1256c Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 8 May 2025 21:14:23 +0300
Subject: [PATCH 1/4] Machinery for grabbing an extended vacuum statistics on
table relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen is a number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible is a number of pages that are marked as all-visible in vm during
vacuum.
wraparound_failsafe_count is a number of times when the vacuum starts urgent cleanup
to prevent wraparound problem which is critical for the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 150 +++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 52 +++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 46 +++-
src/backend/utils/adt/pgstatfuncs.c | 147 ++++++++++++
src/backend/utils/error/elog.c | 13 +
src/backend/utils/misc/guc_tables.c | 9 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 80 +++++-
src/include/utils/elog.h | 1 +
.../vacuum-extending-in-repetable-read.out | 53 ++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 53 ++++
src/test/regress/expected/rules.out | 44 +++-
.../expected/vacuum_tables_statistics.out | 227 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 183 ++++++++++++++
22 files changed, 1095 insertions(+), 16 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f28326bad09..28f222afe60 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -290,6 +290,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -408,6 +409,8 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} LVRelState;
@@ -419,6 +422,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -475,6 +490,106 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
@@ -632,7 +747,14 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -652,7 +774,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -669,6 +791,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -758,6 +881,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->vm_new_visible_frozen_pages = 0;
vacrel->vm_new_frozen_pages = 0;
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
+ vacrel->wraparound_failsafe_count = 0;
/*
* Get cutoffs that determine which deleted tuples are considered DEAD,
@@ -924,6 +1048,26 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ }
+
/*
* Report results to the cumulative stats system, too.
*
@@ -939,7 +1083,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime);
+ starttime,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -2972,6 +3117,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[2] = {0, 0};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count ++;
/*
* Abandon use of a buffer access strategy to allow use of all of
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 745a04ef26e..07623a045fa 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 15efb02badb..0205b9bb58c 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -713,7 +713,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_vacuum_time(C.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1412,3 +1414,51 @@ REVOKE ALL ON pg_aios FROM PUBLIC;
GRANT SELECT ON pg_aios TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_aios() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.vm_new_frozen_pages AS vm_new_frozen_pages,
+ stats.vm_new_visible_pages AS vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ stats.missed_dead_pages AS missed_dead_pages,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.recently_dead_tuples AS recently_dead_tuples,
+ stats.missed_dead_tuples AS missed_dead_tuples,
+
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 33a33bf6b1c..ffb7e1eef4c 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -115,6 +115,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2514,6 +2517,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 2b9d548cdeb..7924c526cb0 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1054,6 +1054,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 8b57845e870..23cb62e36a7 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = true;
/* ----------
* state shared with pgstat_*.c
@@ -260,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -897,7 +896,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -966,7 +964,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1082,7 +1080,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1133,7 +1131,7 @@ pgstat_prep_snapshot(void)
}
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 28587e2916b..ee0385cd809 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -209,7 +211,7 @@ pgstat_drop_relation(Relation rel)
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime)
+ TimestampTz starttime, ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -235,6 +237,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -881,6 +885,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -1004,3 +1011,40 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
+ dst->vm_new_visible_pages += src->vm_new_visible_pages;
+ dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->missed_dead_pages += src->missed_dead_pages;
+ dst->missed_dead_tuples += src->missed_dead_tuples;
+
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 97af7c6554f..416f3c51b0c 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2258,3 +2264,144 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 47af743990f..8c9e8fb18e1 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1624,6 +1624,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 2f8cbd86759..115f0c51cc2 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1508,6 +1508,15 @@ struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
+ gettext_noop("Collects vacuum statistics for table relations."),
+ NULL
+ },
+ &pgstat_track_vacuum_statistics,
+ true,
+ NULL, NULL, NULL
+ },
{
{"track_wal_io_timing", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects timing statistics for WAL I/O activity."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 34826d01380..b4971a9dffe 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -665,6 +665,7 @@
#track_wal_io_timing = off
#track_functions = none # none, pl, all
#stats_fetch_consistency = cache # cache, none, snapshot
+#track_vacuum_statistics = off
# - Monitoring -
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 62beb71da28..e2aaaf6cd59 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12566,4 +12566,22 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int4,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index bc37a80dc74..6d1b2991ce5 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -327,6 +327,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 378f2f2c2ba..6c88d57aef7 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,6 +111,53 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* the number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -153,6 +200,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -211,7 +268,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB7
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB8
typedef struct PgStat_ArchiverStats
{
@@ -375,6 +432,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -453,6 +512,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autovacuum_time;
PgStat_Counter total_analyze_time;
PgStat_Counter total_autoanalyze_time;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -660,7 +724,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime);
+ TimestampTz starttime, ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -711,6 +775,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -799,6 +874,7 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
/*
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 5eac0e16970..6a30a4db47d 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrcode(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..87f7e40b4a6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index e3c669a29c7..6faee3ad2c3 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -96,6 +96,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5893d89573d
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,53 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6cf828ca8d0..10a482e2db4 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1829,7 +1829,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_vacuum_time(c.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2222,7 +2224,9 @@ pg_stat_sys_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2274,9 +2278,43 @@ pg_stat_user_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.missed_dead_pages,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.missed_dead_tuples,
+ stats.wraparound_failsafe,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..b5ea9c9ab1e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,227 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+ relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_scanned | pages_removed | vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | missed_dead_pages | tuples_deleted | tuples_frozen | recently_dead_tuples | missed_dead_tuples | index_vacuum_count | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+---------------+---------------------+----------------------+-----------------------------+-------------------+----------------+---------------+----------------------+--------------------+--------------------+-------------+---------+-----------+---------------+----------------+------------+------------
+ vestat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
+(1 row)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
+---------------------+----------------------+----------------------+-----------------------+-----------------------------
+ 0 | 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | f | f
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index a424be2a6bf..ee0343c2729 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,3 +140,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..5bc34bec64b
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v22-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (57.9K, 4-v22-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 49e728d788435ae909ecf05374506918e1e525f2 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 8 May 2025 21:20:26 +0300
Subject: [PATCH 2/4] Machinery for grabbing an extended vacuum statistics on
index relations.
They are gathered separatelly from table statistics.
As for tables, we gather vacuum shared buffers statistics for index relations like
value of total_blks_hit, total_blks_read, total_blks_dirtied, wal statistics, io time
during flushing buffer pages to disk, delay and total time.
Due to the fact that such statistics are common as for tables, as for indexes we
set them in the union ExtVacReport structure. We only added some determination 'type'
field to highlight what kind belong to these statistics: PGSTAT_EXTVAC_TABLE or
PGSTAT_EXTVAC_INDEX. Generally, PGSTAT_EXTVAC_INVALID type leads to wrong code process.
Some statistics belong only one type of both tables or indexes. So, we added substructures
sych table and index inside ExtVacReport structure.
Therefore, we gather only for tables such statistics like number of scanned, removed pages,
their charecteristics according VM (all-visible and frozen). In addition, for tables we
gather number frozen, deleted and recently dead tuples and how many times vacuum processed
indexes for tables.
Controversally for indexes we gather number of deleted pages and deleted tuples only.
As for tables, deleted pages and deleted tuples reflect the overall performance of the vacuum
for the index relationship.
Since the vacuum cleans up references to tuple indexes before cleaning up table tuples,
which adds some complexity to the vacuum process, namely the vacuum switches from cleaning up
a table to its indexes and back during its operation, we need to save the vacuum statistics
collected for the heap before it starts cleaning up the indexes.
That's why it's necessary to track the vacuum statistics for the heap several times during
the vacuum procedure. To avoid sending the statistics to the Cumulative Statistics System
several times, we save these statistics in the LVRelState structure and only after vacuum
finishes cleaning up the heap, it sends them to the Cumulative Statistics System.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 292 ++++++++++++++----
src/backend/catalog/system_views.sql | 32 ++
src/backend/commands/vacuumparallel.c | 14 +
src/backend/utils/activity/pgstat.c | 4 +
src/backend/utils/activity/pgstat_relation.c | 48 ++-
src/backend/utils/adt/pgstatfuncs.c | 133 +++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 25 ++
src/include/pgstat.h | 58 +++-
.../vacuum-extending-in-repetable-read.out | 4 +-
src/test/regress/expected/rules.out | 22 ++
.../expected/vacuum_index_statistics.out | 183 +++++++++++
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 151 +++++++++
15 files changed, 873 insertions(+), 105 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 28f222afe60..f42c700f6bb 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -291,6 +291,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -411,6 +412,8 @@ typedef struct LVRelState
BlockNumber eager_scan_remaining_fails;
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReport extVacReport;
} LVRelState;
@@ -422,19 +425,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-/*
- * Counters and usage data for extended stats tracking.
- */
-typedef struct LVExtStatCounters
-{
- TimestampTz starttime;
- WalUsage walusage;
- BufferUsage bufusage;
- double VacuumDelayTime;
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-} LVExtStatCounters;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static void heap_vacuum_eager_scan_setup(LVRelState *vacrel,
@@ -556,27 +546,25 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
endtime = GetCurrentTimestamp();
TimestampDifference(counters->starttime, endtime, &secs, &usecs);
- memset(report, 0, sizeof(ExtVacReport));
-
/*
* Fill additional statistics on a vacuum processing operation.
*/
- report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
- report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
- report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
- report->total_blks_written = bufusage.shared_blks_written;
+ report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written += bufusage.shared_blks_written;
- report->wal_records = walusage.wal_records;
- report->wal_fpi = walusage.wal_fpi;
- report->wal_bytes = walusage.wal_bytes;
+ report->wal_records += walusage.wal_records;
+ report->wal_fpi += walusage.wal_fpi;
+ report->wal_bytes += walusage.wal_bytes;
- report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
- report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
- report->total_time = secs * 1000. + usecs / 1000.;
+ report->total_time += secs * 1000. + usecs / 1000.;
if (!rel->pgstat_info || !pgstat_track_counts)
/*
@@ -585,12 +573,96 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
*/
return;
- report->blks_fetched =
+ report->blks_fetched +=
rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
- report->blks_hit =
+ report->blks_hit +=
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Set initial values for common heap and index statistics*/
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ memset(report, 0, sizeof(ExtVacReport));
+
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVExtStatCounters *extVacCounters, LVRelState *vacrel)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReport.type = PGSTAT_EXTVAC_TABLE;
+ vacrel->extVacReport.table.pages_scanned += vacrel->scanned_pages;
+ vacrel->extVacReport.table.pages_removed += vacrel->removed_pages;
+ vacrel->extVacReport.table.vm_new_frozen_pages += vacrel->vm_new_frozen_pages;
+ vacrel->extVacReport.table.vm_new_visible_pages += vacrel->vm_new_visible_pages;
+ vacrel->extVacReport.table.vm_new_visible_frozen_pages += vacrel->vm_new_visible_frozen_pages;
+ vacrel->extVacReport.tuples_deleted += vacrel->tuples_deleted;
+ vacrel->extVacReport.table.tuples_frozen += vacrel->tuples_frozen;
+ vacrel->extVacReport.table.recently_dead_tuples += vacrel->recently_dead_tuples;
+ vacrel->extVacReport.table.recently_dead_tuples += vacrel->recently_dead_tuples;
+ vacrel->extVacReport.table.missed_dead_tuples += vacrel->missed_dead_tuples;
+ vacrel->extVacReport.table.missed_dead_pages += vacrel->missed_dead_pages;
+ vacrel->extVacReport.table.index_vacuum_count += vacrel->num_index_scans;
+ vacrel->extVacReport.table.wraparound_failsafe_count += vacrel->wraparound_failsafe_count;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -748,13 +820,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
LVExtStatCounters extVacCounters;
- ExtVacReport extVacReport;
char **indnames = NULL;
- ExtVacReport allzero;
-
- /* Initialize vacuum statistics */
- memset(&allzero, 0, sizeof(ExtVacReport));
- extVacReport = allzero;
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -774,7 +840,6 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
- extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -961,6 +1026,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
*/
lazy_scan_heap(vacrel);
+ extvac_stats_start(rel, &extVacCounters);
+
/*
* Free resources managed by dead_items_alloc. This ends parallel mode in
* passing when necessary.
@@ -1048,26 +1115,6 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
- /* Make generic extended vacuum stats report */
- extvac_stats_end(rel, &extVacCounters, &extVacReport);
-
- if(pgstat_track_vacuum_statistics)
- {
- /* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
- extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
- extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
- extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
- extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- }
-
/*
* Report results to the cumulative stats system, too.
*
@@ -1077,14 +1124,37 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
* It seems like a good idea to err on the side of not vacuuming again too
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
+ *
+ * We are ready to send vacuum statistics information for heap relations.
*/
- pgstat_report_vacuum(RelationGetRelid(rel),
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
+ vacrel->missed_dead_tuples,
starttime,
- &extVacReport);
+ &(vacrel->extVacReport));
+
+ }
+ else
+ {
+ pgstat_report_vacuum(RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+ }
+
pgstat_progress_end_command();
if (instrument)
@@ -1356,6 +1426,7 @@ lazy_scan_heap(LVRelState *vacrel)
PROGRESS_VACUUM_MAX_DEAD_TUPLE_BYTES
};
int64 initprog_val[3];
+ LVExtStatCounters extVacCounters;
/* Report that we're scanning the heap, advertising total # of blocks */
initprog_val[0] = PROGRESS_VACUUM_PHASE_SCAN_HEAP;
@@ -1370,6 +1441,13 @@ lazy_scan_heap(LVRelState *vacrel)
vacrel->next_unskippable_eager_scanned = false;
vacrel->next_unskippable_vmbuffer = InvalidBuffer;
+ /*
+ * Due to the fact that vacuum heap processing needs their index vacuuming
+ * we need to track them separately and accumulate heap vacuum statistics
+ * separately. So last processes are related to only heap vacuuming process.
+ */
+ extvac_stats_start(vacrel->rel, &extVacCounters);
+
/*
* Set up the read stream for vacuum's first pass through the heap.
*
@@ -1434,8 +1512,26 @@ lazy_scan_heap(LVRelState *vacrel)
/* Perform a round of index and heap vacuuming */
vacrel->consider_bypass_optimization = false;
+
+ /*
+ * Lazy vacuum stage includes index vacuuming and cleaning up stage, so
+ * we prefer tracking them separately.
+ * Before starting to process the indexes save the current heap statistics
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
+
lazy_vacuum(vacrel);
+ /*
+ * After completion lazy vacuum, we start again tracking vacuum statistics for
+ * heap-related objects like FSM, VM, provide heap prunning.
+ * It seems dangerously that we have start tracking but there are no end, but
+ * it is safe. The end tracking is located before lazy vacuum stage in the same
+ * loop or after it.
+ */
+ extvac_stats_start(vacrel->rel, &extVacCounters);
+
/*
* Vacuum the Free Space Map to make newly-freed space visible on
* upper-level FSM pages. Note that blkno is the previously
@@ -1658,6 +1754,12 @@ lazy_scan_heap(LVRelState *vacrel)
read_stream_end(stream);
+ /*
+ * Vacuum can process lazy vacuum again and we save heap statistics now
+ * just in case in tend to avoid collecting vacuum index statistics again.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
/*
* Do index vacuuming (call each index's ambulkdelete routine), then do
* related heap vacuuming
@@ -1665,6 +1767,12 @@ lazy_scan_heap(LVRelState *vacrel)
if (vacrel->dead_items_info->num_items > 0)
lazy_vacuum(vacrel);
+ /*
+ * We need to take into account heap vacuum statistics during process of
+ * FSM.
+ */
+ extvac_stats_start(vacrel->rel, &extVacCounters);
+
/*
* Vacuum the remainder of the Free Space Map. We must do this whether or
* not there were indexes, and whether or not we bypassed index vacuuming.
@@ -1677,6 +1785,10 @@ lazy_scan_heap(LVRelState *vacrel)
/* report all blocks vacuumed */
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, rel_pages);
+ /* Before starting final index clan up stage save heap statistics */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
+
/* Do final index cleanup (call each index's amvacuumcleanup routine) */
if (vacrel->nindexes > 0 && vacrel->do_index_cleanup)
lazy_cleanup_all_indexes(vacrel);
@@ -2594,6 +2706,7 @@ static void
lazy_vacuum(LVRelState *vacrel)
{
bool bypass;
+ LVExtStatCounters extVacCounters;
/* Should not end up here with no indexes */
Assert(vacrel->nindexes > 0);
@@ -2606,6 +2719,9 @@ lazy_vacuum(LVRelState *vacrel)
return;
}
+ /* Set initial statistics values to gather vacuum statistics for the heap */
+ extvac_stats_start(vacrel->rel, &extVacCounters);
+
/*
* Consider bypassing index vacuuming (and heap vacuuming) entirely.
*
@@ -2662,6 +2778,14 @@ lazy_vacuum(LVRelState *vacrel)
TidStoreMemoryUsage(vacrel->dead_items) < 32 * 1024 * 1024);
}
+ /*
+ * Vacuum is likely to vacuum indexes again, so save vacuum statistics for
+ * heap relations now.
+ * The vacuum process below doesn't contain any useful statistics information
+ * for heap if indexes won't be processed, but we will track them separately.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+
if (bypass)
{
/*
@@ -2678,11 +2802,21 @@ lazy_vacuum(LVRelState *vacrel)
}
else if (lazy_vacuum_all_indexes(vacrel))
{
- /*
- * We successfully completed a round of index vacuuming. Do related
- * heap vacuuming now.
- */
- lazy_vacuum_heap_rel(vacrel);
+ /* Now the vacuum is going to process heap relation, so
+ * we need to set intial statistic values for tracking.
+ */
+
+ /* Set initial statistics values to gather vacuum statistics for the heap */
+ extvac_stats_start(vacrel->rel, &extVacCounters);
+
+ /*
+ * We successfully completed a round of index vacuuming. Do related
+ * heap vacuuming now.
+ */
+ lazy_vacuum_heap_rel(vacrel);
+
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
}
else
{
@@ -3229,6 +3363,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3247,6 +3386,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3255,6 +3395,15 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3279,6 +3428,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3298,12 +3452,22 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 0205b9bb58c..b4187e5ad54 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1462,3 +1462,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 7924c526cb0..000388a565f 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,15 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 23cb62e36a7..f5f75aa4264 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1176,6 +1176,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index ee0385cd809..9ee03509490 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -1016,6 +1016,9 @@ static void
pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
bool accumulate_reltype_specific_info)
{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
dst->total_blks_read += src->total_blks_read;
dst->total_blks_hit += src->total_blks_hit;
dst->total_blks_dirtied += src->total_blks_dirtied;
@@ -1031,20 +1034,35 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
- dst->vm_new_visible_pages += src->vm_new_visible_pages;
- dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->missed_dead_pages += src->missed_dead_pages;
- dst->missed_dead_tuples += src->missed_dead_tuples;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->tuples_deleted += src->tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 416f3c51b0c..15fa3de0871 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2372,18 +2372,19 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2402,6 +2403,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 115f0c51cc2..42f4cac5e0e 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1510,7 +1510,7 @@ struct config_bool ConfigureNamesBool[] =
},
{
{"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
- gettext_noop("Collects vacuum statistics for table relations."),
+ gettext_noop("Collects vacuum statistics for relations."),
NULL
},
&pgstat_track_vacuum_statistics,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index e2aaaf6cd59..558b313d0db 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12584,4 +12584,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 6d1b2991ce5..fb134f3402e 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -295,6 +296,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -408,4 +429,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 6c88d57aef7..4def2c60d1d 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,11 +111,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -144,18 +152,44 @@ typedef struct ExtVacReport
double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
double total_time; /* total time of a vacuum operation, in msec */
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
- int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
- int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
- int64 missed_dead_pages; /* pages with missed dead tuples */
int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
- int64 index_vacuum_count; /* the number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 87f7e40b4a6..6d960423912 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -34,7 +34,7 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
(1 row)
step s1_commit: COMMIT;
@@ -48,6 +48,6 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
(1 row)
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 10a482e2db4..4e5e5ca54da 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2283,6 +2283,28 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schemaname,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
rel.relname,
stats.relid,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..e00a0fc683c
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+ relid | schemaname | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+------------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+(0 rows)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ee0343c2729..0197830b5cd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -144,4 +144,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..ae146e1d23f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,151 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] v22-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (31.1K, 5-v22-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From e19e46e20339b477fbcd4538f0b985386f9b7dfa Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 4 Feb 2025 17:57:44 +0300
Subject: [PATCH 3/4] Machinery for grabbing an extended vacuum statistics on
databases.
Database vacuum statistics information is the collected general
vacuum statistics indexes and tables owned by the databases, which
they belong to.
In addition to the fact that there are far fewer databases in a system
than relations, vacuum statistics for a database contain fewer statistics
than relations, but they are enough to indicate that something may be
wrong in the system and prompt the administrator to enable extended
monitoring for relations.
So, buffer, wal, statistics of I/O time of read and writen blocks
statistics will be observed because they are collected for both
tables, indexes. In addition, we show the number of errors caught
during operation of the vacuum only for the error level.
wraparound_failsafe_count is a number of times when the vacuum starts
urgent cleanup to prevent wraparound problem which is critical for
the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 17 ++-
src/backend/catalog/system_views.sql | 27 ++++-
src/backend/utils/activity/pgstat.c | 2 +-
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 46 +++++++-
src/backend/utils/adt/pgstatfuncs.c | 100 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 13 ++-
src/include/pgstat.h | 5 +-
.../vacuum-extending-in-repetable-read.spec | 6 ++
src/test/regress/expected/rules.out | 17 +++
.../expected/vacuum_index_statistics.out | 16 +--
...ut => vacuum_tables_and_db_statistics.out} | 87 +++++++++++++--
src/test/regress/parallel_schedule | 2 +-
.../regress/sql/vacuum_index_statistics.sql | 6 +-
...ql => vacuum_tables_and_db_statistics.sql} | 69 +++++++++++-
16 files changed, 381 insertions(+), 35 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (82%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (81%)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f42c700f6bb..1caf3ac3b36 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -660,7 +660,7 @@ accumulate_heap_vacuum_statistics(LVExtStatCounters *extVacCounters, LVRelState
vacrel->extVacReport.table.missed_dead_tuples += vacrel->missed_dead_tuples;
vacrel->extVacReport.table.missed_dead_pages += vacrel->missed_dead_pages;
vacrel->extVacReport.table.index_vacuum_count += vacrel->num_index_scans;
- vacrel->extVacReport.table.wraparound_failsafe_count += vacrel->wraparound_failsafe_count;
+ vacrel->extVacReport.wraparound_failsafe_count += vacrel->wraparound_failsafe_count;
}
@@ -4080,6 +4080,9 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4095,6 +4098,9 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4110,16 +4116,25 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index b4187e5ad54..0f8346d7b3c 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1493,4 +1493,29 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.errors AS errors
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index f5f75aa4264..85557736a3a 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-bool pgstat_track_vacuum_statistics = true;
+bool pgstat_track_vacuum_statistics = false;
/* ----------
* state shared with pgstat_*.c
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index b31f20d41bc..65207d30378 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -485,6 +485,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 9ee03509490..1695680ea62 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -205,6 +205,38 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.type = m_type;
+ pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.errors++;
+ dbentry->vacuum_ext.type = m_type;
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
@@ -216,6 +248,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -274,6 +307,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
}
/*
@@ -1030,6 +1073,8 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->blk_write_time += src->blk_write_time;
dst->delay_time += src->delay_time;
dst->total_time += src->total_time;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->errors += src->errors;
if (!accumulate_reltype_specific_info)
return;
@@ -1057,7 +1102,6 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->table.index_vacuum_count += src->table.index_vacuum_count;
dst->table.missed_dead_pages += src->table.missed_dead_pages;
dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
}
else if (dst->type == PGSTAT_EXTVAC_INDEX)
{
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 15fa3de0871..c2acdcf0e0e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2383,7 +2383,7 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
@@ -2513,6 +2513,104 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 42f4cac5e0e..a24dec63f3a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1514,7 +1514,7 @@ struct config_bool ConfigureNamesBool[] =
NULL
},
&pgstat_track_vacuum_statistics,
- true,
+ false,
NULL, NULL, NULL
},
{
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 558b313d0db..4710bf997c4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12585,12 +12585,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,wraparound_failsafe,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 4def2c60d1d..f8158aa353c 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -154,6 +154,9 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
+ int32 errors;
+ int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+
ExtVacReportType type; /* heap, index, etc. */
/* ----------
@@ -183,7 +186,6 @@ typedef struct ExtVacReport
int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} table;
struct
{
@@ -762,6 +764,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5893d89573d..cfec3159580 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -18,6 +18,9 @@ teardown
}
session s1
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s1_begin_repeatable_read {
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
select count(ival) from test_vacuum_stat_isolation where id>900;
@@ -25,6 +28,9 @@ step s1_begin_repeatable_read {
step s1_commit { COMMIT; }
session s2
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 4e5e5ca54da..f63f25f94d8 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2283,6 +2283,23 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schemaname,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index e00a0fc683c..9e5d33342c9 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -16,8 +16,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -33,12 +37,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -181,3 +180,4 @@ WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
(1 row)
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 82%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index b5ea9c9ab1e..0300e7b6276 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,7 +6,6 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
--------------
@@ -16,8 +15,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -37,12 +40,12 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -225,3 +228,69 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
(1 row)
DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 0197830b5cd..fa2489716cc 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -145,4 +145,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index ae146e1d23f..9b7e645187d 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -14,8 +14,7 @@ SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -33,7 +32,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -149,3 +148,4 @@ FROM pg_stat_vacuum_indexes vt, pg_class c
WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 81%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 5bc34bec64b..ca7dbde9387 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,15 +7,13 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -36,7 +34,13 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -180,4 +184,59 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+DROP TABLE vestat CASCADE;
+
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
\ No newline at end of file
--
2.34.1
[text/x-patch] v22-0004-Add-documentation-about-the-system-views-that-are-us.patch (24.5K, 6-v22-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From d6ff4cdfc22e2721ff7d1c1e91e340976420e29e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 4/4] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 755 +++++++++++++++++++++++++++++++++
1 file changed, 755 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index b58c52ea50f..7e5acd7c52e 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5474,4 +5474,759 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.34.1
[text/x-patch] vacuum_stats.diff (37.9K, 7-vacuum_stats.diff)
download | inline diff:
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 1caf3ac3b36..8a328638fd3 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -413,7 +413,7 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
} LVRelState;
@@ -525,7 +525,7 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters *counters,
- ExtVacReport *report)
+ PgStat_VacuumRelationCounts *report)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -603,9 +603,9 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
void
extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report)
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report)
{
- memset(report, 0, sizeof(ExtVacReport));
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
extvac_stats_end(rel, &counters->common, report);
report->type = PGSTAT_EXTVAC_INDEX;
@@ -1127,33 +1127,18 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
*
* We are ready to send vacuum statistics information for heap relations.
*/
- if(pgstat_track_vacuum_statistics)
- {
- /* Make generic extended vacuum stats report and
- * fill heap-specific extended stats fields.
- */
- extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
- accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
- pgstat_report_vacuum(RelationGetRelid(rel),
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime,
- &(vacrel->extVacReport));
+ vacrel->missed_dead_tuples,
+ starttime);
- }
- else
- {
- pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime,
- NULL);
- }
+ /* Make generic extended vacuum stats report and fill heap-specific extended stats fields */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &(vacrel->extVacReport));
+ accumulate_heap_vacuum_statistics(&extVacCounters, vacrel);
+ pgstat_report_tab_vacuum_extstats(vacrel->reloid, rel->rd_rel->relisshared, &(vacrel->extVacReport));
pgstat_progress_end_command();
@@ -3364,7 +3349,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3395,14 +3380,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared,
+ &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -3429,7 +3410,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3459,14 +3440,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_cleanup_one_index(&ivinfo, istat);
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared,
+ &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -4081,7 +4058,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -4099,7 +4076,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -4117,7 +4094,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+ pgstat_report_vacuum_error();
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
@@ -4125,7 +4102,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+ pgstat_report_vacuum_error();
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
@@ -4133,7 +4110,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_TRUNCATE:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index fbaed5359ad..72c8e339c45 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1873,6 +1873,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 739a92bdcc1..e4fa754aab4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2327,6 +2327,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 5fbbcdaabb1..c4b910cd928 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1789,6 +1789,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 000388a565f..9401e46d755 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,7 +869,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -909,14 +909,10 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared,
+ &extVacReport);
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 9c2443e1ecd..183f7514d2d 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 85557736a3a..ca764a3a214 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -478,6 +478,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 65207d30378..80e6c7c229a 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
@@ -485,7 +494,6 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
- memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 1695680ea62..acc8f0b8a52 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
@@ -205,50 +203,17 @@ pgstat_drop_relation(Relation rel)
}
}
-/* ---------
- * pgstat_report_vacuum_error() -
- *
- * Tell the collector about an (auto)vacuum interruption.
- * ---------
- */
-void
-pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
-{
- PgStat_EntryRef *entry_ref;
- PgStatShared_Relation *shtabentry;
- PgStat_StatTabEntry *tabentry;
- Oid dboid = MyDatabaseId;
- PgStat_StatDBEntry *dbentry; /* pending database entry */
-
- if (!pgstat_track_counts)
- return;
-
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
- dboid, tableoid, false);
-
- shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
- tabentry = &shtabentry->stats;
-
- tabentry->vacuum_ext.type = m_type;
- pgstat_unlock_entry(entry_ref);
-
- dbentry = pgstat_prep_database_pending(dboid);
- dbentry->vacuum_ext.errors++;
- dbentry->vacuum_ext.type = m_type;
-}
-
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params)
+ TimestampTz starttime)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
- PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -270,8 +235,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
- pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
-
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -307,16 +270,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
-
- if (dboid != InvalidOid)
- {
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dboid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
-
- pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
- pgstat_unlock_entry(entry_ref);
- }
}
/*
@@ -951,6 +904,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
@@ -1053,60 +1012,4 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_updated = trans->updated_pre_truncdrop;
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
-}
-
-static void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info)
-{
- if(!pgstat_track_vacuum_statistics)
- return;
-
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->total_time += src->total_time;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->errors += src->errors;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_TABLE)
- {
- dst->table.pages_scanned += src->table.pages_scanned;
- dst->table.pages_removed += src->table.pages_removed;
- dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
- dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
- dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->table.tuples_frozen += src->table.tuples_frozen;
- dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
- dst->table.index_vacuum_count += src->table.index_vacuum_count;
- dst->table.missed_dead_pages += src->table.missed_dead_pages;
- dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->tuples_deleted += src->tuples_deleted;
- }
- }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c2acdcf0e0e..423a256c83a 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2275,89 +2275,29 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
-
- Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- tabentry = pgstat_fetch_stat_tabentry(relid);
-
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = pending;
i = 0;
@@ -2416,69 +2356,29 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
-
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
+ PgStat_VacuumRelationCounts allzero;
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
-
- tabentry = pgstat_fetch_stat_tabentry(relid);
-
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = pending;
i = 0;
@@ -2523,67 +2423,29 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
Oid dbid = PG_GETARG_OID(0);
- PgStat_StatDBEntry *dbentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumDBCounts *extvacuum;
+ PgStat_VacuumDBCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
-
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
+ PgStat_VacuumDBCounts allzero;
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
- INT4OID, -1, 0);
+ pending = pgstat_fetch_stat_vacuum_dbentry(dbid);
- Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
-
- dbentry = pgstat_fetch_stat_dbentry(dbid);
-
- if (dbentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(dbentry->vacuum_ext);
- }
-
- i = 0;
+ extvacuum = pending;
values[i++] = ObjectIdGetDatum(dbid);
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index fb134f3402e..f895151ca09 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -432,5 +432,5 @@ extern double anl_get_next_S(double t, int n, double *stateptr);
extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx *counters);
extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report);
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f8158aa353c..f57a96d3aa2 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -116,12 +116,60 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_TABLE = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
+ * PgStat_TableCounts The actual per-table counts kept by a backend
+ *
+ * This struct should contain only actual event counters, because we make use
+ * of pg_memory_is_all_zeros() to detect whether there are any stats updates
+ * to apply.
*
- * ExtVacReport
+ * It is a component of PgStat_TableStatus (within-backend state).
+ *
+ * Note: for a table, tuples_returned is the number of tuples successfully
+ * fetched by heap_getnext, while tuples_fetched is the number of tuples
+ * successfully fetched by heap_fetch under the control of bitmap indexscans.
+ * For an index, tuples_returned is the number of index entries returned by
+ * the index AM, while tuples_fetched is the number of tuples successfully
+ * fetched by heap_fetch under the control of simple indexscans for this index.
+ *
+ * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
+ * actions, regardless of whether the transaction committed. delta_live_tuples,
+ * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
+ * Note that delta_live_tuples and delta_dead_tuples can be negative!
+ * ----------
+ */
+typedef struct PgStat_TableCounts
+{
+ PgStat_Counter numscans;
+
+ PgStat_Counter tuples_returned;
+ PgStat_Counter tuples_fetched;
+
+ PgStat_Counter tuples_inserted;
+ PgStat_Counter tuples_updated;
+ PgStat_Counter tuples_deleted;
+ PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_newpage_updated;
+ bool truncdropped;
+
+ PgStat_Counter delta_live_tuples;
+ PgStat_Counter delta_dead_tuples;
+ PgStat_Counter changed_tuples;
+
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+} PgStat_TableCounts;
+
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
*
* Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
@@ -129,7 +177,7 @@ typedef enum ExtVacReportType
* pages_deleted refer to free space within the index file
* ----------
*/
-typedef struct ExtVacReport
+typedef struct PgStat_VacuumRelationCounts
{
/* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
int64 total_blks_read;
@@ -154,7 +202,6 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
- int32 errors;
int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
ExtVacReportType type; /* heap, index, etc. */
@@ -192,61 +239,44 @@ typedef struct ExtVacReport
int64 pages_deleted; /* number of pages deleted by vacuum */
} index;
} /* per_type_stats */;
-} ExtVacReport;
+} PgStat_VacuumRelationCounts;
-/* ----------
- * PgStat_TableCounts The actual per-table counts kept by a backend
- *
- * This struct should contain only actual event counters, because we make use
- * of pg_memory_is_all_zeros() to detect whether there are any stats updates
- * to apply.
- *
- * It is a component of PgStat_TableStatus (within-backend state).
- *
- * Note: for a table, tuples_returned is the number of tuples successfully
- * fetched by heap_getnext, while tuples_fetched is the number of tuples
- * successfully fetched by heap_fetch under the control of bitmap indexscans.
- * For an index, tuples_returned is the number of index entries returned by
- * the index AM, while tuples_fetched is the number of tuples successfully
- * fetched by heap_fetch under the control of simple indexscans for this index.
- *
- * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
- * actions, regardless of whether the transaction committed. delta_live_tuples,
- * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
- * Note that delta_live_tuples and delta_dead_tuples can be negative!
- * ----------
- */
-typedef struct PgStat_TableCounts
+typedef struct PgStat_VacuumRelationStatus
{
- PgStat_Counter numscans;
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
- PgStat_Counter tuples_returned;
- PgStat_Counter tuples_fetched;
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
- PgStat_Counter tuples_inserted;
- PgStat_Counter tuples_updated;
- PgStat_Counter tuples_deleted;
- PgStat_Counter tuples_hot_updated;
- PgStat_Counter tuples_newpage_updated;
- bool truncdropped;
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
- PgStat_Counter delta_live_tuples;
- PgStat_Counter delta_dead_tuples;
- PgStat_Counter changed_tuples;
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
- PgStat_Counter rev_all_visible_pages;
- PgStat_Counter rev_all_frozen_pages;
+ int64 tuples_deleted; /* tuples deleted by vacuum */
- /*
- * Additional cumulative stat on vacuum operations.
- * Use an expensive structure as an abstraction for different types of
- * relations.
- */
- ExtVacReport vacuum_ext;
-} PgStat_TableCounts;
+ int32 errors;
+ int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+} PgStat_VacuumDBCounts;
/* ----------
* PgStat_TableStatus Per-table status within a backend
@@ -272,6 +302,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -468,8 +504,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
-
- ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -551,8 +585,6 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter rev_all_visible_pages;
PgStat_Counter rev_all_frozen_pages;
-
- ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -760,11 +792,11 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params);
+ TimestampTz starttime);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
-extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
+extern void pgstat_report_vacuum_error();
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -895,6 +927,15 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+pgstat_report_tab_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts *pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts *pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid);
+PgStat_VacuumDBCounts *pgstat_fetch_stat_vacuum_dbentry(Oid dbid);
/*
* Functions in pgstat_wal.c
*/
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index d5557e6e998..140adbcdbd6 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -439,6 +439,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -607,6 +619,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index f44169fd5a3..454661f9d6a 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-05-09 12:31 Alena Rybakina <[email protected]>
parent: Andrei Lepikhov <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2025-05-09 12:31 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; Alexander Korotkov <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; jian he <[email protected]>
Hi!
On 22.04.2025 21:23, Andrei Lepikhov wrote:
> On 10/28/24 14:40, Alexander Korotkov wrote:
>> On Sun, Aug 25, 2024 at 6:59 PM Alena Rybakina
>>> If I missed something or misunderstood, can you explain in more detail?
>>
>> Actually, I mean why do we need a possibility to return statistics for
>> all tables/indexes in one function call? User anyway is supposed to
>> use pg_stat_vacuum_indexes/pg_stat_vacuum_tables view, which do
>> function calls one per relation. I suppose we can get rid of
>> possibility to get all the objects in one function call and just
>> return a tuple from the functions like other pgstatfuncs.c functions
>> do.
> I suppose it was designed this way because databases may contain
> thousands of tables and indexes - remember, at least, partitions. But
> it may be okay to use the SRF_FIRSTCALL_INIT / SRF_RETURN_NEXT API. I
> think by registering a prosupport routine predicting cost and rows of
> these calls, we may let the planner build adequate plans for queries
> involving those stats - people will definitely join it with something
> else in the database.
>
I think we can add this, but first we need to answer the main question -
are there cases when we have statistics for a relation that are not in
pg_class? After all, we have views that show vacuum statistics for all
relations for objects stored in pg_class.
+CREATE VIEW pg_stat_vacuum_tables AS
...
FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL*pg_stat_get_vacuum_tables(rel.oid)* stats
+WHERE rel.relkind = 'r';
I tend to think that such a case will happen because to solve the
problem with the memory consumed for storing vacuum statistics, we need
to store them separately from the relations' statistics (I already wrote
the code here [0]), so
the approach with the output of all statistics from a snapshot, as we
did here [1] and removed this approach here [2] and this approach now
makes sense and it is worth organizing it as you suggest.
I can add the code if no one is against it.
[0]
https://www.postgresql.org/message-id/2a04ad18-5572-4633-848b-eb57209e7ac0%40postgrespro.ru
[1]
https://www.postgresql.org/message-id/995657bc-9966-47c0-b085-4c5e8886d249%40postgrespro.ru
[2]
https://www.postgresql.org/message-id/CAPpHfdvSo3mfH%3D2m4ADCHAuN%3D22SnBY3TrPaPbGKTw3r_Jaw7Q%40mail...
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-05-12 12:30 Amit Kapila <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 2 replies; 77+ messages in thread
From: Amit Kapila @ 2025-05-12 12:30 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
On Fri, May 9, 2025 at 5:34 PM Alena Rybakina <[email protected]> wrote:
>
> I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and
> this allows us to solve the problem with the memory consumed.
>
I think this patch is trying to collect data similar to what we do for
pg_stat_statements for SQL statements. So, can't we follow a similar
idea such that these additional statistics will be collected once some
external module like pg_stat_statements is enabled? That module should
be responsible for accumulating and resetting the data, so we won't
have this memory consumption issue.
BTW, how will these new statistics be used to autotune a vacuum? And
do we need all the statistics proposed by this patch?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-05-13 09:49 Alena Rybakina <[email protected]>
parent: Amit Kapila <[email protected]>
1 sibling, 2 replies; 77+ messages in thread
From: Alena Rybakina @ 2025-05-13 09:49 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
Hi!
On 12.05.2025 08:30, Amit Kapila wrote:
> On Fri, May 9, 2025 at 5:34 PM Alena Rybakina <[email protected]> wrote:
>> I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and
>> this allows us to solve the problem with the memory consumed.
>>
> I think this patch is trying to collect data similar to what we do for
> pg_stat_statements for SQL statements. So, can't we follow a similar
> idea such that these additional statistics will be collected once some
> external module like pg_stat_statements is enabled? That module should
> be responsible for accumulating and resetting the data, so we won't
> have this memory consumption issue.
The idea is good, it will require one hook for the pgstat_report_vacuum
function, the extvac_stats_start and extvac_stats_end functions can be
run if the extension is loaded, so as not to add more hooks.
But I see a problem here with tracking deleted objects for which
statistics are no longer needed. There are two solutions to this and I
don't like both of them, to be honest.
The first way is to add a background process that will go through the
table with saved statistics and check whether the relation or the
database are relevant now or not and if not, then
delete the vacuum statistics information for it. This may be
resource-intensive. The second way is to add hooks for deleting the
database and relationships (functions dropdb, index_drop,
heap_drop_with_catalog).
> BTW, how will these new statistics be used to autotune a vacuum?
yes, but they are collected on demand - by guc.
> And
> do we need all the statistics proposed by this patch?
>
Regarding this issue, it was discussed here and so far we have come to
the conclusion that statistics are needed for a deep understanding of
the work of vacuum statistics [0] [1] [2].
[0]
https://www.postgresql.org/message-id/0B6CBF4C-CC2A-4200-9126-CE3A390D938B%40upgrade.com
[1]
https://www.postgresql.org/message-id/6732acf8ce0f31025b535ae1a64568750924a887.camel%40moonset.ru
[2]
https://www.postgresql.org/message-id/5AA8FFD5-6DE2-4A31-8E00-AE98F738F5D1%40upgrade.com
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-05-14 08:55 Amit Kapila <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 0 replies; 77+ messages in thread
From: Amit Kapila @ 2025-05-14 08:55 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
On Tue, May 13, 2025 at 3:19 PM Alena Rybakina
<[email protected]> wrote:
>
> On 12.05.2025 08:30, Amit Kapila wrote:
> > On Fri, May 9, 2025 at 5:34 PM Alena Rybakina <[email protected]> wrote:
> >> I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and
> >> this allows us to solve the problem with the memory consumed.
> >>
> > I think this patch is trying to collect data similar to what we do for
> > pg_stat_statements for SQL statements. So, can't we follow a similar
> > idea such that these additional statistics will be collected once some
> > external module like pg_stat_statements is enabled? That module should
> > be responsible for accumulating and resetting the data, so we won't
> > have this memory consumption issue.
> The idea is good, it will require one hook for the pgstat_report_vacuum
> function, the extvac_stats_start and extvac_stats_end functions can be
> run if the extension is loaded, so as not to add more hooks.
> But I see a problem here with tracking deleted objects for which
> statistics are no longer needed. There are two solutions to this and I
> don't like both of them, to be honest.
> The first way is to add a background process that will go through the
> table with saved statistics and check whether the relation or the
> database are relevant now or not and if not, then
> delete the vacuum statistics information for it. This may be
> resource-intensive. The second way is to add hooks for deleting the
> database and relationships (functions dropdb, index_drop,
> heap_drop_with_catalog).
>
How does pg_stat_io manages this? I mean how it removes objects that
are dropped? Does some background task removes it?
> > BTW, how will these new statistics be used to autotune a vacuum?
> yes, but they are collected on demand - by guc.
> > And
> > do we need all the statistics proposed by this patch?
> >
> Regarding this issue, it was discussed here and so far we have come to
> the conclusion that statistics are needed for a deep understanding of
> the work of vacuum statistics [0] [1] [2].
>
I haven't gone through the emails, but my opinion is to break the
number of stats into some important subset of stats first and then
keep enhancing it. Right now, the patch struggles with two concerns:
one is what the design should be to capture the required stats, and
the second is convincing ourselves whether we need all the stats it is
trying to expose. Breaking into a smaller subset of stats could
alleviate the second concern.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-06-02 16:25 Alexander Korotkov <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 2 replies; 77+ messages in thread
From: Alexander Korotkov @ 2025-06-02 16:25 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Amit Kapila <[email protected]>; pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
On Tue, May 13, 2025 at 12:49 PM Alena Rybakina
<[email protected]> wrote:
> On 12.05.2025 08:30, Amit Kapila wrote:
> > On Fri, May 9, 2025 at 5:34 PM Alena Rybakina <[email protected]> wrote:
> >> I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and
> >> this allows us to solve the problem with the memory consumed.
> >>
> > I think this patch is trying to collect data similar to what we do for
> > pg_stat_statements for SQL statements. So, can't we follow a similar
> > idea such that these additional statistics will be collected once some
> > external module like pg_stat_statements is enabled? That module should
> > be responsible for accumulating and resetting the data, so we won't
> > have this memory consumption issue.
> The idea is good, it will require one hook for the pgstat_report_vacuum
> function, the extvac_stats_start and extvac_stats_end functions can be
> run if the extension is loaded, so as not to add more hooks.
+1
Nice idea of a hook. Given the volume of the patch, it might be a
good idea to keep this as an extension.
> But I see a problem here with tracking deleted objects for which
> statistics are no longer needed. There are two solutions to this and I
> don't like both of them, to be honest.
> The first way is to add a background process that will go through the
> table with saved statistics and check whether the relation or the
> database are relevant now or not and if not, then
> delete the vacuum statistics information for it. This may be
> resource-intensive. The second way is to add hooks for deleting the
> database and relationships (functions dropdb, index_drop,
> heap_drop_with_catalog).
Can we workaround this with object_access_hook?
------
Regards,
Alexander Korotkov
Supabase
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-06-02 16:50 Alena Rybakina <[email protected]>
parent: Alexander Korotkov <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2025-06-02 16:50 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Amit Kapila <[email protected]>; pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
On 02.06.2025 19:25, Alexander Korotkov wrote:
> On Tue, May 13, 2025 at 12:49 PM Alena Rybakina
> <[email protected]> wrote:
>> On 12.05.2025 08:30, Amit Kapila wrote:
>>> On Fri, May 9, 2025 at 5:34 PM Alena Rybakina <[email protected]> wrote:
>>>> I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and
>>>> this allows us to solve the problem with the memory consumed.
>>>>
>>> I think this patch is trying to collect data similar to what we do for
>>> pg_stat_statements for SQL statements. So, can't we follow a similar
>>> idea such that these additional statistics will be collected once some
>>> external module like pg_stat_statements is enabled? That module should
>>> be responsible for accumulating and resetting the data, so we won't
>>> have this memory consumption issue.
>> The idea is good, it will require one hook for the pgstat_report_vacuum
>> function, the extvac_stats_start and extvac_stats_end functions can be
>> run if the extension is loaded, so as not to add more hooks.
> +1
> Nice idea of a hook. Given the volume of the patch, it might be a
> good idea to keep this as an extension.
Okay, I'll realize it and apply the patch)
>
>> But I see a problem here with tracking deleted objects for which
>> statistics are no longer needed. There are two solutions to this and I
>> don't like both of them, to be honest.
>> The first way is to add a background process that will go through the
>> table with saved statistics and check whether the relation or the
>> database are relevant now or not and if not, then
>> delete the vacuum statistics information for it. This may be
>> resource-intensive. The second way is to add hooks for deleting the
>> database and relationships (functions dropdb, index_drop,
>> heap_drop_with_catalog).
> Can we workaround this with object_access_hook?
I think this could fix the problem. For the OAT-DROP access type, we can
call a function to reset the vacuum statistics for relations that are
about to be dropped.
At the moment, I don’t see any limitations to using this approach.
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-06-02 19:56 Alena Rybakina <[email protected]>
parent: Alexander Korotkov <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2025-06-02 19:56 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Amit Kapila <[email protected]>; pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
On 02.06.2025 19:25, Alexander Korotkov wrote:
> On Tue, May 13, 2025 at 12:49 PM Alena Rybakina
> <[email protected]> wrote:
>> On 12.05.2025 08:30, Amit Kapila wrote:
>>> On Fri, May 9, 2025 at 5:34 PM Alena Rybakina <[email protected]> wrote:
>>>> I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and
>>>> this allows us to solve the problem with the memory consumed.
>>>>
>>> I think this patch is trying to collect data similar to what we do for
>>> pg_stat_statements for SQL statements. So, can't we follow a similar
>>> idea such that these additional statistics will be collected once some
>>> external module like pg_stat_statements is enabled? That module should
>>> be responsible for accumulating and resetting the data, so we won't
>>> have this memory consumption issue.
>> The idea is good, it will require one hook for the pgstat_report_vacuum
>> function, the extvac_stats_start and extvac_stats_end functions can be
>> run if the extension is loaded, so as not to add more hooks.
> +1
> Nice idea of a hook. Given the volume of the patch, it might be a
> good idea to keep this as an extension.
Today, I finalized the vacuum statistics separation approach and
refactored the vacuum statistics structures (patch 4).
I also reworked the table statistics to avoid mixing index statistics in
parallel vacuum mode (patch 2).
The new approach excludes buffer usage and WAL statistics for indexes
from the table’s statistics.
For timing, if vacuuming is sequential, the total time spent on all
indexes is subtracted from the table’s total vacuum time by adding up
the individual index vacuum times. If vacuuming is parallel, the total
index vacuum time is subtracted as a whole.
static void
accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport
*extVacIdxStats)
{
if (!pgstat_track_vacuum_statistics)
return;
/* Fill heap-specific extended stats fields */
vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
vacrel->extVacReportIdx.blk_write_time +=
extVacIdxStats->blk_write_time;
vacrel->extVacReportIdx.total_blks_dirtied +=
extVacIdxStats->total_blks_dirtied;
vacrel->extVacReportIdx.total_blks_hit +=
extVacIdxStats->total_blks_hit;
vacrel->extVacReportIdx.total_blks_read +=
extVacIdxStats->total_blks_read;
vacrel->extVacReportIdx.total_blks_written +=
extVacIdxStats->total_blks_written;
vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
}
if (ParallelVacuumIsActive(vacrel))
{
LVExtStatCounters counters;
ExtVacReport extVacReport;
memset(&extVacReport, 0, sizeof(ExtVacReport));
extvac_stats_start(vacrel->rel, &counters);
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
extvac_stats_end(vacrel->rel, &counters, &extVacReport);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
Currently, database statistics work incorrectly — I'm investigating the
issue.
In parallel, I'm starting work on the extension.
--
Regards,
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] 0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (71.3K, 2-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 6c52152b3d96d4b7dc2de2692b116af20be67dca Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 8 May 2025 21:14:23 +0300
Subject: [PATCH 1/5] Machinery for grabbing an extended vacuum statistics on
table relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen is a number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible is a number of pages that are marked as all-visible in vm during
vacuum.
wraparound_failsafe_count is a number of times when the vacuum starts urgent cleanup
to prevent wraparound problem which is critical for the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 150 +++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 52 +++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 46 +++-
src/backend/utils/adt/pgstatfuncs.c | 147 ++++++++++++
src/backend/utils/error/elog.c | 13 +
src/backend/utils/misc/guc_tables.c | 9 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 80 +++++-
src/include/utils/elog.h | 1 +
.../vacuum-extending-in-repetable-read.out | 53 ++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 53 ++++
src/test/regress/expected/rules.out | 44 +++-
.../expected/vacuum_tables_statistics.out | 227 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 183 ++++++++++++++
22 files changed, 1095 insertions(+), 16 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 708674d8fcf..ee27e70a798 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -290,6 +290,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -408,6 +409,8 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} LVRelState;
@@ -419,6 +422,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -475,6 +490,106 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
@@ -632,7 +747,14 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -652,7 +774,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -669,6 +791,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -758,6 +881,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->vm_new_visible_frozen_pages = 0;
vacrel->vm_new_frozen_pages = 0;
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
+ vacrel->wraparound_failsafe_count = 0;
/*
* Get cutoffs that determine which deleted tuples are considered DEAD,
@@ -924,6 +1048,26 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ }
+
/*
* Report results to the cumulative stats system, too.
*
@@ -939,7 +1083,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime);
+ starttime,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -2977,6 +3122,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[2] = {0, 0};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count ++;
/*
* Abandon use of a buffer access strategy to allow use of all of
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 745a04ef26e..07623a045fa 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 08f780a2e63..47d27314b55 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -708,7 +708,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_vacuum_time(C.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1407,3 +1409,51 @@ REVOKE ALL ON pg_aios FROM PUBLIC;
GRANT SELECT ON pg_aios TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_aios() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.vm_new_frozen_pages AS vm_new_frozen_pages,
+ stats.vm_new_visible_pages AS vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ stats.missed_dead_pages AS missed_dead_pages,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.recently_dead_tuples AS recently_dead_tuples,
+ stats.missed_dead_tuples AS missed_dead_tuples,
+
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 33a33bf6b1c..ffb7e1eef4c 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -115,6 +115,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2514,6 +2517,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 0feea1d30ec..2b55d9b7c0e 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1054,6 +1054,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 8b57845e870..23cb62e36a7 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = true;
/* ----------
* state shared with pgstat_*.c
@@ -260,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -897,7 +896,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -966,7 +964,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1082,7 +1080,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1133,7 +1131,7 @@ pgstat_prep_snapshot(void)
}
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 28587e2916b..ee0385cd809 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -209,7 +211,7 @@ pgstat_drop_relation(Relation rel)
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime)
+ TimestampTz starttime, ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -235,6 +237,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -881,6 +885,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -1004,3 +1011,40 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
+ dst->vm_new_visible_pages += src->vm_new_visible_pages;
+ dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->missed_dead_pages += src->missed_dead_pages;
+ dst->missed_dead_tuples += src->missed_dead_tuples;
+
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index e980109f245..a5610199893 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2258,3 +2264,144 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 47af743990f..8c9e8fb18e1 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1624,6 +1624,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 2f8cbd86759..115f0c51cc2 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1508,6 +1508,15 @@ struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
+ gettext_noop("Collects vacuum statistics for table relations."),
+ NULL
+ },
+ &pgstat_track_vacuum_statistics,
+ true,
+ NULL, NULL, NULL
+ },
{
{"track_wal_io_timing", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects timing statistics for WAL I/O activity."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 87ce76b18f4..e971b390281 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -661,6 +661,7 @@
#track_wal_io_timing = off
#track_functions = none # none, pl, all
#stats_fetch_consistency = cache # cache, none, snapshot
+#track_vacuum_statistics = off
# - Monitoring -
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 37a484147a8..c04d3880241 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12556,4 +12556,22 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int4,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index bc37a80dc74..6d1b2991ce5 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -327,6 +327,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 378f2f2c2ba..6c88d57aef7 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,6 +111,53 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* the number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -153,6 +200,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -211,7 +268,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB7
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB8
typedef struct PgStat_ArchiverStats
{
@@ -375,6 +432,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -453,6 +512,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autovacuum_time;
PgStat_Counter total_analyze_time;
PgStat_Counter total_autoanalyze_time;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -660,7 +724,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime);
+ TimestampTz starttime, ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -711,6 +775,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -799,6 +874,7 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
/*
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 5eac0e16970..6a30a4db47d 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrcode(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..87f7e40b4a6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index e3c669a29c7..6faee3ad2c3 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -96,6 +96,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5893d89573d
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,53 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6cf828ca8d0..10a482e2db4 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1829,7 +1829,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_vacuum_time(c.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2222,7 +2224,9 @@ pg_stat_sys_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2274,9 +2278,43 @@ pg_stat_user_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.missed_dead_pages,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.missed_dead_tuples,
+ stats.wraparound_failsafe,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..b5ea9c9ab1e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,227 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+ relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_scanned | pages_removed | vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | missed_dead_pages | tuples_deleted | tuples_frozen | recently_dead_tuples | missed_dead_tuples | index_vacuum_count | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+---------------+---------------------+----------------------+-----------------------------+-------------------+----------------+---------------+----------------------+--------------------+--------------------+-------------+---------+-----------+---------------+----------------+------------+------------
+ vestat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
+(1 row)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
+---------------------+----------------------+----------------------+-----------------------+-----------------------------
+ 0 | 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | f | f
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index a424be2a6bf..ee0343c2729 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,3 +140,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..5bc34bec64b
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] 0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (55.2K, 3-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 4e19e818679ae56608a0bc087fdc463b3d2183fb Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Jun 2025 19:35:02 +0300
Subject: [PATCH 2/5] Machinery for grabbing an extended vacuum statistics on
index relations.
They are gathered separatelly from table statistics.
As for tables, we gather vacuum shared buffers statistics for index relations like
value of total_blks_hit, total_blks_read, total_blks_dirtied, wal statistics, io time
during flushing buffer pages to disk, delay and total time.
Due to the fact that such statistics are common as for tables, as for indexes we
set them in the union ExtVacReport structure. We only added some determination 'type'
field to highlight what kind belong to these statistics: PGSTAT_EXTVAC_TABLE or
PGSTAT_EXTVAC_INDEX. Generally, PGSTAT_EXTVAC_INVALID type leads to wrong code process.
Some statistics belong only one type of both tables or indexes. So, we added substructures
sych table and index inside ExtVacReport structure.
Therefore, we gather only for tables such statistics like number of scanned, removed pages,
their charecteristics according VM (all-visible and frozen). In addition, for tables we
gather number frozen, deleted and recently dead tuples and how many times vacuum processed
indexes for tables.
Controversally for indexes we gather number of deleted pages and deleted tuples only.
As for tables, deleted pages and deleted tuples reflect the overall performance of the vacuum
for the index relationship.
Since the vacuum cleans up references to tuple indexes before cleaning up table tuples,
which adds some complexity to the vacuum process, namely the vacuum switches from cleaning up
a table to its indexes and back during its operation, we need to save the vacuum statistics
collected for the heap before it starts cleaning up the indexes.
That's why it's necessary to track the vacuum statistics for the heap several times during
the vacuum procedure. To avoid sending the statistics to the Cumulative Statistics System
several times, we save these statistics in the LVRelState structure and only after vacuum
finishes cleaning up the heap, it sends them to the Cumulative Statistics System.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 270 ++++++++++++++----
src/backend/catalog/system_views.sql | 32 +++
src/backend/commands/vacuumparallel.c | 14 +
src/backend/utils/activity/pgstat.c | 4 +
src/backend/utils/activity/pgstat_relation.c | 48 +++-
src/backend/utils/adt/pgstatfuncs.c | 133 ++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 25 ++
src/include/pgstat.h | 58 +++-
.../vacuum-extending-in-repetable-read.out | 4 +-
src/test/regress/expected/rules.out | 22 ++
.../expected/vacuum_index_statistics.out | 183 ++++++++++++
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 151 ++++++++++
15 files changed, 864 insertions(+), 92 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index ee27e70a798..0888be2afea 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -291,6 +291,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -411,6 +412,8 @@ typedef struct LVRelState
BlockNumber eager_scan_remaining_fails;
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReport extVacReportIdx;
} LVRelState;
@@ -422,19 +425,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-/*
- * Counters and usage data for extended stats tracking.
- */
-typedef struct LVExtStatCounters
-{
- TimestampTz starttime;
- WalUsage walusage;
- BufferUsage bufusage;
- double VacuumDelayTime;
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-} LVExtStatCounters;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static void heap_vacuum_eager_scan_setup(LVRelState *vacrel,
@@ -556,27 +546,25 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
endtime = GetCurrentTimestamp();
TimestampDifference(counters->starttime, endtime, &secs, &usecs);
- memset(report, 0, sizeof(ExtVacReport));
-
/*
* Fill additional statistics on a vacuum processing operation.
*/
- report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
- report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
- report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
- report->total_blks_written = bufusage.shared_blks_written;
+ report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written += bufusage.shared_blks_written;
- report->wal_records = walusage.wal_records;
- report->wal_fpi = walusage.wal_fpi;
- report->wal_bytes = walusage.wal_bytes;
+ report->wal_records += walusage.wal_records;
+ report->wal_fpi += walusage.wal_fpi;
+ report->wal_bytes += walusage.wal_bytes;
- report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
- report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
- report->total_time = secs * 1000. + usecs / 1000.;
+ report->total_time += secs * 1000. + usecs / 1000.;
if (!rel->pgstat_info || !pgstat_track_counts)
/*
@@ -585,12 +573,131 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
*/
return;
- report->blks_fetched =
+ report->blks_fetched +=
rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
- report->blks_hit =
+ report->blks_hit +=
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Set initial values for common heap and index statistics*/
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ memset(report, 0, sizeof(ExtVacReport));
+
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
+ extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
+ extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
+ extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
+ extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
+ extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
+ extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
+ extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
+ extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+
+ extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
+ extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+
+}
+
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacIdxStats)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
+ vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
+ vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
+ vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
+ vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
+ vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
+ vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
+ vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
+ vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
+ vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
+
+ vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -750,11 +857,9 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
LVExtStatCounters extVacCounters;
ExtVacReport extVacReport;
char **indnames = NULL;
- ExtVacReport allzero;
/* Initialize vacuum statistics */
- memset(&allzero, 0, sizeof(ExtVacReport));
- extVacReport = allzero;
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -800,6 +905,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
vac_open_indexes(vacrel->rel, RowExclusiveLock, &vacrel->nindexes,
@@ -1051,23 +1158,6 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
/* Make generic extended vacuum stats report */
extvac_stats_end(rel, &extVacCounters, &extVacReport);
- if(pgstat_track_vacuum_statistics)
- {
- /* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
- extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
- extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
- extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
- extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- }
-
/*
* Report results to the cumulative stats system, too.
*
@@ -1078,13 +1168,34 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(RelationGetRelid(rel),
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
+ vacrel->missed_dead_tuples,
starttime,
&extVacReport);
+
+ }
+ else
+ {
+ pgstat_report_vacuum(RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+ }
+
pgstat_progress_end_command();
if (instrument)
@@ -2781,10 +2892,20 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
* that parallel VACUUM only gets the precheck and this postcheck.
@@ -3205,10 +3326,20 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
estimated_count);
+
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
/* Reset the progress counters */
@@ -3234,6 +3365,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3252,6 +3388,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3260,6 +3397,19 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3284,6 +3434,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3303,12 +3458,25 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 47d27314b55..83d55e78606 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1457,3 +1457,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 2b55d9b7c0e..65de45a4447 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,15 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 23cb62e36a7..f5f75aa4264 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1176,6 +1176,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index ee0385cd809..9ee03509490 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -1016,6 +1016,9 @@ static void
pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
bool accumulate_reltype_specific_info)
{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
dst->total_blks_read += src->total_blks_read;
dst->total_blks_hit += src->total_blks_hit;
dst->total_blks_dirtied += src->total_blks_dirtied;
@@ -1031,20 +1034,35 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
- dst->vm_new_visible_pages += src->vm_new_visible_pages;
- dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->missed_dead_pages += src->missed_dead_pages;
- dst->missed_dead_tuples += src->missed_dead_tuples;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->tuples_deleted += src->tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index a5610199893..482929b75e9 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2372,18 +2372,19 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2402,6 +2403,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 115f0c51cc2..42f4cac5e0e 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1510,7 +1510,7 @@ struct config_bool ConfigureNamesBool[] =
},
{
{"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
- gettext_noop("Collects vacuum statistics for table relations."),
+ gettext_noop("Collects vacuum statistics for relations."),
NULL
},
&pgstat_track_vacuum_statistics,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c04d3880241..8c77ae96100 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12574,4 +12574,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 6d1b2991ce5..fb134f3402e 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -295,6 +296,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -408,4 +429,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 6c88d57aef7..4def2c60d1d 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,11 +111,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -144,18 +152,44 @@ typedef struct ExtVacReport
double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
double total_time; /* total time of a vacuum operation, in msec */
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
- int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
- int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
- int64 missed_dead_pages; /* pages with missed dead tuples */
int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
- int64 index_vacuum_count; /* the number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 87f7e40b4a6..6d960423912 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -34,7 +34,7 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
(1 row)
step s1_commit: COMMIT;
@@ -48,6 +48,6 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
(1 row)
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 10a482e2db4..4e5e5ca54da 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2283,6 +2283,28 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schemaname,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
rel.relname,
stats.relid,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..e00a0fc683c
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+ relid | schemaname | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+------------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+(0 rows)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ee0343c2729..0197830b5cd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -144,4 +144,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..ae146e1d23f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,151 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] 0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (31.2K, 4-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 2978fa59fc553b1a711731ae83159e4f44241dee Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 4 Feb 2025 17:57:44 +0300
Subject: [PATCH 3/5] Machinery for grabbing an extended vacuum statistics on
databases.
Database vacuum statistics information is the collected general
vacuum statistics indexes and tables owned by the databases, which
they belong to.
In addition to the fact that there are far fewer databases in a system
than relations, vacuum statistics for a database contain fewer statistics
than relations, but they are enough to indicate that something may be
wrong in the system and prompt the administrator to enable extended
monitoring for relations.
So, buffer, wal, statistics of I/O time of read and writen blocks
statistics will be observed because they are collected for both
tables, indexes. In addition, we show the number of errors caught
during operation of the vacuum only for the error level.
wraparound_failsafe_count is a number of times when the vacuum starts
urgent cleanup to prevent wraparound problem which is critical for
the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 17 ++-
src/backend/catalog/system_views.sql | 27 ++++-
src/backend/utils/activity/pgstat.c | 2 +-
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 46 +++++++-
src/backend/utils/adt/pgstatfuncs.c | 100 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 13 ++-
src/include/pgstat.h | 5 +-
.../vacuum-extending-in-repetable-read.spec | 6 ++
src/test/regress/expected/rules.out | 17 +++
.../expected/vacuum_index_statistics.out | 16 +--
...ut => vacuum_tables_and_db_statistics.out} | 87 +++++++++++++--
src/test/regress/parallel_schedule | 2 +-
.../regress/sql/vacuum_index_statistics.sql | 6 +-
...ql => vacuum_tables_and_db_statistics.sql} | 69 +++++++++++-
16 files changed, 381 insertions(+), 35 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (82%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (81%)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 0888be2afea..3d72f74b05e 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -660,7 +660,7 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
@@ -4089,6 +4089,9 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4104,6 +4107,9 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4119,16 +4125,25 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 83d55e78606..0ae31b87989 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1488,4 +1488,29 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.errors AS errors
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index f5f75aa4264..85557736a3a 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-bool pgstat_track_vacuum_statistics = true;
+bool pgstat_track_vacuum_statistics = false;
/* ----------
* state shared with pgstat_*.c
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index b31f20d41bc..65207d30378 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -485,6 +485,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 9ee03509490..1695680ea62 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -205,6 +205,38 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.type = m_type;
+ pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.errors++;
+ dbentry->vacuum_ext.type = m_type;
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
@@ -216,6 +248,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -274,6 +307,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
}
/*
@@ -1030,6 +1073,8 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->blk_write_time += src->blk_write_time;
dst->delay_time += src->delay_time;
dst->total_time += src->total_time;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->errors += src->errors;
if (!accumulate_reltype_specific_info)
return;
@@ -1057,7 +1102,6 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->table.index_vacuum_count += src->table.index_vacuum_count;
dst->table.missed_dead_pages += src->table.missed_dead_pages;
dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
}
else if (dst->type == PGSTAT_EXTVAC_INDEX)
{
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 482929b75e9..a2ece2c36cf 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2383,7 +2383,7 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
@@ -2513,6 +2513,104 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 42f4cac5e0e..a24dec63f3a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1514,7 +1514,7 @@ struct config_bool ConfigureNamesBool[] =
NULL
},
&pgstat_track_vacuum_statistics,
- true,
+ false,
NULL, NULL, NULL
},
{
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8c77ae96100..4e1e29eec7e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12575,12 +12575,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,wraparound_failsafe,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 4def2c60d1d..f8158aa353c 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -154,6 +154,9 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
+ int32 errors;
+ int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+
ExtVacReportType type; /* heap, index, etc. */
/* ----------
@@ -183,7 +186,6 @@ typedef struct ExtVacReport
int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} table;
struct
{
@@ -762,6 +764,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5893d89573d..cfec3159580 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -18,6 +18,9 @@ teardown
}
session s1
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s1_begin_repeatable_read {
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
select count(ival) from test_vacuum_stat_isolation where id>900;
@@ -25,6 +28,9 @@ step s1_begin_repeatable_read {
step s1_commit { COMMIT; }
session s2
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 4e5e5ca54da..f63f25f94d8 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2283,6 +2283,23 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schemaname,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index e00a0fc683c..9e5d33342c9 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -16,8 +16,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -33,12 +37,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -181,3 +180,4 @@ WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
(1 row)
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 82%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index b5ea9c9ab1e..0300e7b6276 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,7 +6,6 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
--------------
@@ -16,8 +15,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -37,12 +40,12 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -225,3 +228,69 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
(1 row)
DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 0197830b5cd..fa2489716cc 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -145,4 +145,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index ae146e1d23f..9b7e645187d 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -14,8 +14,7 @@ SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -33,7 +32,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -149,3 +148,4 @@ FROM pg_stat_vacuum_indexes vt, pg_class c
WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 81%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 5bc34bec64b..ca7dbde9387 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,15 +7,13 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -36,7 +34,13 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -180,4 +184,59 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+DROP TABLE vestat CASCADE;
+
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
\ No newline at end of file
--
2.34.1
[text/x-patch] 0004-Vacuum-statistics-have-been-separated-from-regular-r.patch (63.6K, 5-0004-Vacuum-statistics-have-been-separated-from-regular-r.patch)
download | inline diff:
From e2da3e08e08f3c1edc12160a237f15516e506270 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Jun 2025 22:24:38 +0300
Subject: [PATCH 4/5] Vacuum statistics have been separated from regular
relation and database statistics to reduce memory usage. Dedicated
PGSTAT_KIND_VACUUM_RELATION and PGSTAT_KIND_VACUUM_DB entries were added to
the stats collector to efficiently allocate memory for vacuum-specific
metrics, which require significantly more space per relation.
---
src/backend/access/heap/vacuumlazy.c | 168 +++++------
src/backend/catalog/heap.c | 1 +
src/backend/catalog/index.c | 1 +
src/backend/commands/dbcommands.c | 1 +
src/backend/commands/vacuumparallel.c | 14 +-
src/backend/utils/activity/Makefile | 1 +
src/backend/utils/activity/pgstat.c | 28 ++
src/backend/utils/activity/pgstat_database.c | 10 +-
src/backend/utils/activity/pgstat_relation.c | 111 +-------
src/backend/utils/activity/pgstat_vacuum.c | 224 +++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 285 +++++--------------
src/include/commands/vacuum.h | 2 +-
src/include/pgstat.h | 200 +++++++------
src/include/utils/pgstat_internal.h | 15 +
src/include/utils/pgstat_kind.h | 4 +-
15 files changed, 569 insertions(+), 496 deletions(-)
create mode 100644 src/backend/utils/activity/pgstat_vacuum.c
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3d72f74b05e..24b6b64c3fc 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -413,7 +413,7 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
- ExtVacReport extVacReportIdx;
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -525,7 +525,7 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters *counters,
- ExtVacReport *report)
+ PgStat_VacuumRelationCounts *report)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -549,22 +549,22 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
/*
* Fill additional statistics on a vacuum processing operation.
*/
- report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
- report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
- report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
- report->total_blks_written += bufusage.shared_blks_written;
+ report->common.total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->common.total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->common.total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->common.total_blks_written += bufusage.shared_blks_written;
- report->wal_records += walusage.wal_records;
- report->wal_fpi += walusage.wal_fpi;
- report->wal_bytes += walusage.wal_bytes;
+ report->common.wal_records += walusage.wal_records;
+ report->common.wal_fpi += walusage.wal_fpi;
+ report->common.wal_bytes += walusage.wal_bytes;
- report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
- report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
- report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
- report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
- report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
+ report->common.blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->common.blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->common.blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->common.blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->common.delay_time += VacuumDelayTime - counters->VacuumDelayTime;
- report->total_time += secs * 1000. + usecs / 1000.;
+ report->common.total_time += secs * 1000. + usecs / 1000.;
if (!rel->pgstat_info || !pgstat_track_counts)
/*
@@ -573,9 +573,9 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
*/
return;
- report->blks_fetched +=
+ report->common.blks_fetched +=
rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
- report->blks_hit +=
+ report->common.blks_hit +=
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
@@ -603,9 +603,9 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
void
extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report)
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report)
{
- memset(report, 0, sizeof(ExtVacReport));
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
extvac_stats_end(rel, &counters->common, report);
report->type = PGSTAT_EXTVAC_INDEX;
@@ -618,7 +618,7 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
*/
/* Fill index-specific extended stats fields */
- report->tuples_deleted =
+ report->common.tuples_deleted =
stats->tuples_removed - counters->tuples_removed;
report->index.pages_deleted =
stats->pages_deleted - counters->pages_deleted;
@@ -641,7 +641,7 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
* procudure.
*/
static void
-accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacStats)
{
if (!pgstat_track_vacuum_statistics)
return;
@@ -653,49 +653,49 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
- extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
- extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
- extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
- extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
- extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
- extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
- extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
- extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
- extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
- extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
}
static void
-accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacIdxStats)
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacIdxStats)
{
if (!pgstat_track_vacuum_statistics)
return;
/* Fill heap-specific extended stats fields */
- vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
- vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
- vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
- vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
- vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
- vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
- vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
- vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
- vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
- vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
-
- vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
}
@@ -855,11 +855,11 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
LVExtStatCounters extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
char **indnames = NULL;
/* Initialize vacuum statistics */
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -905,7 +905,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
- memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+ memset(&vacrel->extVacReportIdx, 0, sizeof(PgStat_VacuumRelationCounts));
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
@@ -1176,25 +1176,16 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime,
- &extVacReport);
+ pgstat_report_tab_vacuum_extstats(vacrel->reloid, rel->rd_rel->relisshared, &extVacReport);
}
- else
- {
- pgstat_report_vacuum(RelationGetRelid(rel),
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime,
- NULL);
- }
+ starttime);
pgstat_progress_end_command();
@@ -2893,9 +2884,9 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -3327,9 +3318,9 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -3366,7 +3357,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3405,9 +3396,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
if (!ParallelVacuumIsActive(vacrel))
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
}
/* Revert to the previous phase information for error traceback */
@@ -3435,7 +3424,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3472,9 +3461,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
if (!ParallelVacuumIsActive(vacrel))
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
}
/* Revert to the previous phase information for error traceback */
@@ -4075,6 +4062,27 @@ update_relstats_all_indexes(LVRelState *vacrel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+static void
+pgstat_report_vacuum_error()
+{
+ PgStat_VacuumDBCounts *vacuum_dbentry;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ vacuum_dbentry = pgstat_fetch_stat_vacuum_dbentry(MyDatabaseId);
+
+ if(vacuum_dbentry == NULL)
+ return;
+ vacuum_dbentry->errors++;
+}
+
/*
* Error context callback for errors occurring during vacuum. The error
* context messages for index phases should match the messages set in parallel
@@ -4090,7 +4098,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -4108,7 +4116,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -4126,7 +4134,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+ pgstat_report_vacuum_error();
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
@@ -4134,7 +4142,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+ pgstat_report_vacuum_error();
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
@@ -4142,7 +4150,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_TRUNCATE:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index fbaed5359ad..72c8e339c45 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1873,6 +1873,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 739a92bdcc1..e4fa754aab4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2327,6 +2327,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 5fbbcdaabb1..c4b910cd928 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1789,6 +1789,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 65de45a4447..b67326eafcc 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,7 +869,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -909,14 +909,10 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_tab_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared,
+ &extVacReport);
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 9c2443e1ecd..183f7514d2d 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 85557736a3a..ca764a3a214 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -478,6 +478,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 65207d30378..80e6c7c229a 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
@@ -485,7 +494,6 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
- memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 1695680ea62..acc8f0b8a52 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
@@ -205,50 +203,17 @@ pgstat_drop_relation(Relation rel)
}
}
-/* ---------
- * pgstat_report_vacuum_error() -
- *
- * Tell the collector about an (auto)vacuum interruption.
- * ---------
- */
-void
-pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
-{
- PgStat_EntryRef *entry_ref;
- PgStatShared_Relation *shtabentry;
- PgStat_StatTabEntry *tabentry;
- Oid dboid = MyDatabaseId;
- PgStat_StatDBEntry *dbentry; /* pending database entry */
-
- if (!pgstat_track_counts)
- return;
-
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
- dboid, tableoid, false);
-
- shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
- tabentry = &shtabentry->stats;
-
- tabentry->vacuum_ext.type = m_type;
- pgstat_unlock_entry(entry_ref);
-
- dbentry = pgstat_prep_database_pending(dboid);
- dbentry->vacuum_ext.errors++;
- dbentry->vacuum_ext.type = m_type;
-}
-
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params)
+ TimestampTz starttime)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
- PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -270,8 +235,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
- pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
-
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -307,16 +270,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
-
- if (dboid != InvalidOid)
- {
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dboid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
-
- pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
- pgstat_unlock_entry(entry_ref);
- }
}
/*
@@ -951,6 +904,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
@@ -1053,60 +1012,4 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_updated = trans->updated_pre_truncdrop;
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
-}
-
-static void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info)
-{
- if(!pgstat_track_vacuum_statistics)
- return;
-
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->total_time += src->total_time;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->errors += src->errors;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_TABLE)
- {
- dst->table.pages_scanned += src->table.pages_scanned;
- dst->table.pages_removed += src->table.pages_removed;
- dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
- dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
- dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->table.tuples_frozen += src->table.tuples_frozen;
- dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
- dst->table.index_vacuum_count += src->table.index_vacuum_count;
- dst->table.missed_dead_pages += src->table.missed_dead_pages;
- dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->tuples_deleted += src->tuples_deleted;
- }
- }
}
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat_vacuum.c b/src/backend/utils/activity/pgstat_vacuum.c
new file mode 100644
index 00000000000..7e1db137a18
--- /dev/null
+++ b/src/backend/utils/activity/pgstat_vacuum.c
@@ -0,0 +1,224 @@
+#include "postgres.h"
+
+#include "pgstat.h"
+#include "utils/pgstat_internal.h"
+#include "utils/memutils.h"
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+bool pgstat_track_vacuum_statistics_for_relations = false;
+
+#define ACCUMULATE_FIELD(field) dst->field += src->field;
+
+#define ACCUMULATE_SUBFIELD(substruct, field) \
+ (dst->substruct.field += src->substruct.field)
+
+static void
+pgstat_accumulate_common(PgStat_CommonCounts *dst, const PgStat_CommonCounts *src)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->wal_records += src->wal_records;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_bytes += src->wal_bytes;
+
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+}
+
+static void
+pgstat_accumulate_extvac_stats_relations(PgStat_VacuumRelationCounts *dst, PgStat_VacuumRelationCounts *src)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB && src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ dst->common.blks_fetched += src->common.blks_fetched;
+ dst->common.blks_hit += src->common.blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ ACCUMULATE_SUBFIELD(table, pages_scanned);
+ ACCUMULATE_SUBFIELD(table, pages_removed);
+ ACCUMULATE_SUBFIELD(table, vm_new_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_frozen_pages);
+ dst->common.tuples_deleted += src->common.tuples_deleted;
+ ACCUMULATE_SUBFIELD(table, tuples_frozen);
+ ACCUMULATE_SUBFIELD(table, recently_dead_tuples);
+ ACCUMULATE_SUBFIELD(table, index_vacuum_count);
+ ACCUMULATE_SUBFIELD(table, missed_dead_pages);
+ ACCUMULATE_SUBFIELD(table, missed_dead_tuples);
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ ACCUMULATE_SUBFIELD(index, pages_deleted);
+ dst->common.tuples_deleted += src->common.tuples_deleted;
+ }
+}
+
+static void
+pgstat_accumulate_extvac_stats_db(PgStat_VacuumDBCounts *dst, PgStat_VacuumDBCounts *src)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ pgstat_accumulate_common((PgStat_CommonCounts *) dst, (PgStat_CommonCounts *) src);
+ dst->errors += src->errors;
+}
+
+static void
+pgstat_increment_tab_extvac_stats_to_db(PgStat_VacuumDBCounts *dst, PgStat_VacuumRelationCounts *src)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ pgstat_accumulate_common((PgStat_CommonCounts *) dst, (PgStat_CommonCounts *) src);
+}
+
+/*
+ * Report that the table was just vacuumed and flush statistics.
+ */
+void
+pgstat_report_tab_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_VacuumRelation *shtabentry;
+ PgStatShared_VacuumDB *shdbentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_RELATION,
+ dboid, tableoid, false);
+ shtabentry = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+ pgstat_accumulate_extvac_stats_relations(&shtabentry->stats, params);
+
+ pgstat_unlock_entry(entry_ref);
+
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_DB,
+ dboid, InvalidOid, false);
+
+ shdbentry = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ pgstat_increment_tab_extvac_stats_to_db(&shdbentry->stats, params);
+
+ pgstat_unlock_entry(entry_ref);
+}
+
+/*
+ * Flush out pending stats for the entry
+ *
+ * If nowait is true, this function returns false if lock could not
+ * immediately acquired, otherwise true is returned.
+ */
+bool
+pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumRelation *shtabstats;
+ PgStat_RelationVacuumPending *pendingent; /* table entry of shared stats */
+
+ pendingent = (PgStat_RelationVacuumPending *) entry_ref->pending;
+ shtabstats = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts.
+ */
+ if (pg_memory_is_all_zeros(&pendingent,
+ sizeof(struct PgStat_RelationVacuumPending)))
+ return true;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ {
+ return false;
+ }
+
+ pgstat_accumulate_extvac_stats_relations(&(shtabstats->stats), &(pendingent->counts));
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the vacuum collected statistics for one relation or NULL.
+ */
+PgStat_VacuumRelationCounts *
+pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid)
+{
+ return (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_RELATION, dbid, relid);
+}
+
+PgStat_VacuumDBCounts *
+pgstat_fetch_stat_vacuum_dbentry(Oid dbid)
+{
+ return (PgStat_VacuumDBCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_DB, dbid, InvalidOid);
+}
+
+bool
+pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumDB *sharedent;
+ PgStat_VacuumDBCounts *pendingent;
+
+ pendingent = (PgStat_VacuumDBCounts *) entry_ref->pending;
+ sharedent = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ /* The entry was successfully flushed, add the same to database stats */
+ pgstat_accumulate_extvac_stats_db(&(sharedent->stats), pendingent);
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Find or create a local PgStat_VacuumDBCounts entry for dboid.
+ */
+PgStat_VacuumDBCounts *
+pgstat_prep_vacuum_database_pending(Oid dboid)
+{
+ PgStat_EntryRef *entry_ref;
+
+ /*
+ * This should not report stats on database objects before having
+ * connected to a database.
+ */
+ Assert(!OidIsValid(dboid) || OidIsValid(MyDatabaseId));
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_VACUUM_DB, dboid, InvalidOid,
+ NULL);
+
+ if(entry_ref == NULL)
+ return NULL;
+
+ return entry_ref->pending;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index a2ece2c36cf..8603d4dd576 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2265,7 +2265,6 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
-
/*
* Get the vacuum statistics for the heap tables.
*/
@@ -2275,102 +2274,42 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
-
- Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- tabentry = pgstat_fetch_stat_tabentry(relid);
-
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = pending;
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
@@ -2378,28 +2317,28 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
@@ -2416,100 +2355,60 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
-
- Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
-
- tabentry = pgstat_fetch_stat_tabentry(relid);
-
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = pending;
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
@@ -2523,90 +2422,52 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
Oid dbid = PG_GETARG_OID(0);
- PgStat_StatDBEntry *dbentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumDBCounts *extvacuum;
+ PgStat_VacuumDBCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
-
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
+ PgStat_VacuumDBCounts allzero;
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
- INT4OID, -1, 0);
+ pending = pgstat_fetch_stat_vacuum_dbentry(dbid);
- Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
-
- dbentry = pgstat_fetch_stat_dbentry(dbid);
-
- if (dbentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(dbentry->vacuum_ext);
- }
-
- i = 0;
+ extvacuum = pending;
values[i++] = ObjectIdGetDatum(dbid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
values[i++] = Int32GetDatum(extvacuum->errors);
Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index fb134f3402e..f895151ca09 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -432,5 +432,5 @@ extern double anl_get_next_S(double t, int n, double *stateptr);
extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx *counters);
extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report);
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f8158aa353c..a12590948fb 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -116,46 +116,100 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_TABLE = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
+ * PgStat_TableCounts The actual per-table counts kept by a backend
*
- * ExtVacReport
+ * This struct should contain only actual event counters, because we make use
+ * of pg_memory_is_all_zeros() to detect whether there are any stats updates
+ * to apply.
*
- * Additional statistics of vacuum processing over a relation.
- * pages_removed is the amount by which the physically shrank,
- * if any (ie the change in its total size on disk)
- * pages_deleted refer to free space within the index file
+ * It is a component of PgStat_TableStatus (within-backend state).
+ *
+ * Note: for a table, tuples_returned is the number of tuples successfully
+ * fetched by heap_getnext, while tuples_fetched is the number of tuples
+ * successfully fetched by heap_fetch under the control of bitmap indexscans.
+ * For an index, tuples_returned is the number of index entries returned by
+ * the index AM, while tuples_fetched is the number of tuples successfully
+ * fetched by heap_fetch under the control of simple indexscans for this index.
+ *
+ * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
+ * actions, regardless of whether the transaction committed. delta_live_tuples,
+ * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
+ * Note that delta_live_tuples and delta_dead_tuples can be negative!
* ----------
*/
-typedef struct ExtVacReport
+typedef struct PgStat_TableCounts
{
- /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
- int64 total_blks_read;
- int64 total_blks_hit;
- int64 total_blks_dirtied;
- int64 total_blks_written;
+ PgStat_Counter numscans;
- /* blocks missed and hit for just the heap during a vacuum of specific relation */
- int64 blks_fetched;
- int64 blks_hit;
+ PgStat_Counter tuples_returned;
+ PgStat_Counter tuples_fetched;
- /* Vacuum WAL usage stats */
- int64 wal_records; /* wal usage: number of WAL records */
- int64 wal_fpi; /* wal usage: number of WAL full page images produced */
- uint64 wal_bytes; /* wal usage: size of WAL records produced */
+ PgStat_Counter tuples_inserted;
+ PgStat_Counter tuples_updated;
+ PgStat_Counter tuples_deleted;
+ PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_newpage_updated;
+ bool truncdropped;
+
+ PgStat_Counter delta_live_tuples;
+ PgStat_Counter delta_dead_tuples;
+ PgStat_Counter changed_tuples;
+
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
- /* Time stats. */
- double blk_read_time; /* time spent reading pages, in msec */
- double blk_write_time; /* time spent writing pages, in msec */
- double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
- double total_time; /* total time of a vacuum operation, in msec */
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+} PgStat_TableCounts;
- int64 tuples_deleted; /* tuples deleted by vacuum */
+typedef struct PgStat_CommonCounts
+{
+ /* blocks */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* heap blocks */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* WAL */
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+
+ /* Time */
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+
+ /* tuples */
+ int64 tuples_deleted;
+
+ /* failsafe */
+ int32 wraparound_failsafe_count;
+} PgStat_CommonCounts;
- int32 errors;
- int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
ExtVacReportType type; /* heap, index, etc. */
@@ -174,16 +228,16 @@ typedef struct ExtVacReport
{
struct
{
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 pages_scanned; /* heap pages examined (not skipped by VM) */
int64 pages_removed; /* heap pages removed by vacuum "truncation" */
int64 pages_frozen; /* pages marked in VM as frozen */
int64 pages_all_visible; /* pages marked in VM as all-visible */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
} table;
@@ -192,61 +246,21 @@ typedef struct ExtVacReport
int64 pages_deleted; /* number of pages deleted by vacuum */
} index;
} /* per_type_stats */;
-} ExtVacReport;
+} PgStat_VacuumRelationCounts;
-/* ----------
- * PgStat_TableCounts The actual per-table counts kept by a backend
- *
- * This struct should contain only actual event counters, because we make use
- * of pg_memory_is_all_zeros() to detect whether there are any stats updates
- * to apply.
- *
- * It is a component of PgStat_TableStatus (within-backend state).
- *
- * Note: for a table, tuples_returned is the number of tuples successfully
- * fetched by heap_getnext, while tuples_fetched is the number of tuples
- * successfully fetched by heap_fetch under the control of bitmap indexscans.
- * For an index, tuples_returned is the number of index entries returned by
- * the index AM, while tuples_fetched is the number of tuples successfully
- * fetched by heap_fetch under the control of simple indexscans for this index.
- *
- * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
- * actions, regardless of whether the transaction committed. delta_live_tuples,
- * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
- * Note that delta_live_tuples and delta_dead_tuples can be negative!
- * ----------
- */
-typedef struct PgStat_TableCounts
+typedef struct PgStat_VacuumRelationStatus
{
- PgStat_Counter numscans;
-
- PgStat_Counter tuples_returned;
- PgStat_Counter tuples_fetched;
-
- PgStat_Counter tuples_inserted;
- PgStat_Counter tuples_updated;
- PgStat_Counter tuples_deleted;
- PgStat_Counter tuples_hot_updated;
- PgStat_Counter tuples_newpage_updated;
- bool truncdropped;
-
- PgStat_Counter delta_live_tuples;
- PgStat_Counter delta_dead_tuples;
- PgStat_Counter changed_tuples;
-
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-
- PgStat_Counter rev_all_visible_pages;
- PgStat_Counter rev_all_frozen_pages;
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
- /*
- * Additional cumulative stat on vacuum operations.
- * Use an expensive structure as an abstraction for different types of
- * relations.
- */
- ExtVacReport vacuum_ext;
-} PgStat_TableCounts;
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ PgStat_CommonCounts common;
+ int32 errors;
+} PgStat_VacuumDBCounts;
/* ----------
* PgStat_TableStatus Per-table status within a backend
@@ -272,6 +286,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -468,8 +488,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
-
- ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -551,8 +569,6 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter rev_all_visible_pages;
PgStat_Counter rev_all_frozen_pages;
-
- ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -760,11 +776,10 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params);
+ TimestampTz starttime);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
-extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -895,6 +910,15 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+pgstat_report_tab_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts *pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts *pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid);
+PgStat_VacuumDBCounts *pgstat_fetch_stat_vacuum_dbentry(Oid dbid);
/*
* Functions in pgstat_wal.c
*/
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index d5557e6e998..140adbcdbd6 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -439,6 +439,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -607,6 +619,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index f44169fd5a3..454661f9d6a 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
--
2.34.1
[text/x-patch] 0005-Add-documentation-about-the-system-views-that-are-us.patch (24.5K, 6-0005-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 66ddd445242f0181edb903ed7ace60e75ca70890 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 5/5] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 755 +++++++++++++++++++++++++++++++++
1 file changed, 755 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index b58c52ea50f..7e5acd7c52e 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5474,4 +5474,759 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.34.1
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-06-03 12:27 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2025-06-03 12:27 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Amit Kapila <[email protected]>; pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
On 02.06.2025 22:56, Alena Rybakina wrote:
> Today, I finalized the vacuum statistics separation approach and
> refactored the vacuum statistics structures (patch 4).
>
> Currently, database statistics work incorrectly — I'm investigating
> the issue.
I fixed it and attached the patches.
> In parallel, I'm starting work on the extension.
>
I started working on this and will attach the first version soon.
--
Regards,
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] v23-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (71.3K, 2-v23-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 6c52152b3d96d4b7dc2de2692b116af20be67dca Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 8 May 2025 21:14:23 +0300
Subject: [PATCH 1/5] Machinery for grabbing an extended vacuum statistics on
table relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen is a number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible is a number of pages that are marked as all-visible in vm during
vacuum.
wraparound_failsafe_count is a number of times when the vacuum starts urgent cleanup
to prevent wraparound problem which is critical for the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 150 +++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 52 +++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 46 +++-
src/backend/utils/adt/pgstatfuncs.c | 147 ++++++++++++
src/backend/utils/error/elog.c | 13 +
src/backend/utils/misc/guc_tables.c | 9 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 80 +++++-
src/include/utils/elog.h | 1 +
.../vacuum-extending-in-repetable-read.out | 53 ++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 53 ++++
src/test/regress/expected/rules.out | 44 +++-
.../expected/vacuum_tables_statistics.out | 227 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 183 ++++++++++++++
22 files changed, 1095 insertions(+), 16 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 708674d8fcf..ee27e70a798 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -290,6 +290,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -408,6 +409,8 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} LVRelState;
@@ -419,6 +422,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -475,6 +490,106 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
@@ -632,7 +747,14 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -652,7 +774,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -669,6 +791,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -758,6 +881,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->vm_new_visible_frozen_pages = 0;
vacrel->vm_new_frozen_pages = 0;
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
+ vacrel->wraparound_failsafe_count = 0;
/*
* Get cutoffs that determine which deleted tuples are considered DEAD,
@@ -924,6 +1048,26 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ }
+
/*
* Report results to the cumulative stats system, too.
*
@@ -939,7 +1083,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime);
+ starttime,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -2977,6 +3122,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[2] = {0, 0};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count ++;
/*
* Abandon use of a buffer access strategy to allow use of all of
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 745a04ef26e..07623a045fa 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 08f780a2e63..47d27314b55 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -708,7 +708,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_vacuum_time(C.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1407,3 +1409,51 @@ REVOKE ALL ON pg_aios FROM PUBLIC;
GRANT SELECT ON pg_aios TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_aios() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.vm_new_frozen_pages AS vm_new_frozen_pages,
+ stats.vm_new_visible_pages AS vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ stats.missed_dead_pages AS missed_dead_pages,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.recently_dead_tuples AS recently_dead_tuples,
+ stats.missed_dead_tuples AS missed_dead_tuples,
+
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 33a33bf6b1c..ffb7e1eef4c 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -115,6 +115,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2514,6 +2517,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 0feea1d30ec..2b55d9b7c0e 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1054,6 +1054,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 8b57845e870..23cb62e36a7 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = true;
/* ----------
* state shared with pgstat_*.c
@@ -260,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -897,7 +896,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -966,7 +964,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1082,7 +1080,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1133,7 +1131,7 @@ pgstat_prep_snapshot(void)
}
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 28587e2916b..ee0385cd809 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -209,7 +211,7 @@ pgstat_drop_relation(Relation rel)
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime)
+ TimestampTz starttime, ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -235,6 +237,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -881,6 +885,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -1004,3 +1011,40 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
+ dst->vm_new_visible_pages += src->vm_new_visible_pages;
+ dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->missed_dead_pages += src->missed_dead_pages;
+ dst->missed_dead_tuples += src->missed_dead_tuples;
+
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index e980109f245..a5610199893 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2258,3 +2264,144 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 47af743990f..8c9e8fb18e1 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1624,6 +1624,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 2f8cbd86759..115f0c51cc2 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1508,6 +1508,15 @@ struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
+ gettext_noop("Collects vacuum statistics for table relations."),
+ NULL
+ },
+ &pgstat_track_vacuum_statistics,
+ true,
+ NULL, NULL, NULL
+ },
{
{"track_wal_io_timing", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects timing statistics for WAL I/O activity."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 87ce76b18f4..e971b390281 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -661,6 +661,7 @@
#track_wal_io_timing = off
#track_functions = none # none, pl, all
#stats_fetch_consistency = cache # cache, none, snapshot
+#track_vacuum_statistics = off
# - Monitoring -
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 37a484147a8..c04d3880241 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12556,4 +12556,22 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int4,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index bc37a80dc74..6d1b2991ce5 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -327,6 +327,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 378f2f2c2ba..6c88d57aef7 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,6 +111,53 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* the number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -153,6 +200,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -211,7 +268,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB7
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB8
typedef struct PgStat_ArchiverStats
{
@@ -375,6 +432,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -453,6 +512,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autovacuum_time;
PgStat_Counter total_analyze_time;
PgStat_Counter total_autoanalyze_time;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -660,7 +724,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime);
+ TimestampTz starttime, ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -711,6 +775,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -799,6 +874,7 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
/*
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 5eac0e16970..6a30a4db47d 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrcode(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..87f7e40b4a6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index e3c669a29c7..6faee3ad2c3 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -96,6 +96,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5893d89573d
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,53 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6cf828ca8d0..10a482e2db4 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1829,7 +1829,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_vacuum_time(c.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2222,7 +2224,9 @@ pg_stat_sys_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2274,9 +2278,43 @@ pg_stat_user_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.missed_dead_pages,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.missed_dead_tuples,
+ stats.wraparound_failsafe,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..b5ea9c9ab1e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,227 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+ relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_scanned | pages_removed | vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | missed_dead_pages | tuples_deleted | tuples_frozen | recently_dead_tuples | missed_dead_tuples | index_vacuum_count | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+---------------+---------------------+----------------------+-----------------------------+-------------------+----------------+---------------+----------------------+--------------------+--------------------+-------------+---------+-----------+---------------+----------------+------------+------------
+ vestat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
+(1 row)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
+---------------------+----------------------+----------------------+-----------------------+-----------------------------
+ 0 | 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | f | f
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index a424be2a6bf..ee0343c2729 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,3 +140,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..5bc34bec64b
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v23-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (55.2K, 3-v23-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 4e19e818679ae56608a0bc087fdc463b3d2183fb Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Jun 2025 19:35:02 +0300
Subject: [PATCH 2/5] Machinery for grabbing an extended vacuum statistics on
index relations.
They are gathered separatelly from table statistics.
As for tables, we gather vacuum shared buffers statistics for index relations like
value of total_blks_hit, total_blks_read, total_blks_dirtied, wal statistics, io time
during flushing buffer pages to disk, delay and total time.
Due to the fact that such statistics are common as for tables, as for indexes we
set them in the union ExtVacReport structure. We only added some determination 'type'
field to highlight what kind belong to these statistics: PGSTAT_EXTVAC_TABLE or
PGSTAT_EXTVAC_INDEX. Generally, PGSTAT_EXTVAC_INVALID type leads to wrong code process.
Some statistics belong only one type of both tables or indexes. So, we added substructures
sych table and index inside ExtVacReport structure.
Therefore, we gather only for tables such statistics like number of scanned, removed pages,
their charecteristics according VM (all-visible and frozen). In addition, for tables we
gather number frozen, deleted and recently dead tuples and how many times vacuum processed
indexes for tables.
Controversally for indexes we gather number of deleted pages and deleted tuples only.
As for tables, deleted pages and deleted tuples reflect the overall performance of the vacuum
for the index relationship.
Since the vacuum cleans up references to tuple indexes before cleaning up table tuples,
which adds some complexity to the vacuum process, namely the vacuum switches from cleaning up
a table to its indexes and back during its operation, we need to save the vacuum statistics
collected for the heap before it starts cleaning up the indexes.
That's why it's necessary to track the vacuum statistics for the heap several times during
the vacuum procedure. To avoid sending the statistics to the Cumulative Statistics System
several times, we save these statistics in the LVRelState structure and only after vacuum
finishes cleaning up the heap, it sends them to the Cumulative Statistics System.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 270 ++++++++++++++----
src/backend/catalog/system_views.sql | 32 +++
src/backend/commands/vacuumparallel.c | 14 +
src/backend/utils/activity/pgstat.c | 4 +
src/backend/utils/activity/pgstat_relation.c | 48 +++-
src/backend/utils/adt/pgstatfuncs.c | 133 ++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 25 ++
src/include/pgstat.h | 58 +++-
.../vacuum-extending-in-repetable-read.out | 4 +-
src/test/regress/expected/rules.out | 22 ++
.../expected/vacuum_index_statistics.out | 183 ++++++++++++
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 151 ++++++++++
15 files changed, 864 insertions(+), 92 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index ee27e70a798..0888be2afea 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -291,6 +291,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -411,6 +412,8 @@ typedef struct LVRelState
BlockNumber eager_scan_remaining_fails;
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReport extVacReportIdx;
} LVRelState;
@@ -422,19 +425,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-/*
- * Counters and usage data for extended stats tracking.
- */
-typedef struct LVExtStatCounters
-{
- TimestampTz starttime;
- WalUsage walusage;
- BufferUsage bufusage;
- double VacuumDelayTime;
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-} LVExtStatCounters;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static void heap_vacuum_eager_scan_setup(LVRelState *vacrel,
@@ -556,27 +546,25 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
endtime = GetCurrentTimestamp();
TimestampDifference(counters->starttime, endtime, &secs, &usecs);
- memset(report, 0, sizeof(ExtVacReport));
-
/*
* Fill additional statistics on a vacuum processing operation.
*/
- report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
- report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
- report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
- report->total_blks_written = bufusage.shared_blks_written;
+ report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written += bufusage.shared_blks_written;
- report->wal_records = walusage.wal_records;
- report->wal_fpi = walusage.wal_fpi;
- report->wal_bytes = walusage.wal_bytes;
+ report->wal_records += walusage.wal_records;
+ report->wal_fpi += walusage.wal_fpi;
+ report->wal_bytes += walusage.wal_bytes;
- report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
- report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
- report->total_time = secs * 1000. + usecs / 1000.;
+ report->total_time += secs * 1000. + usecs / 1000.;
if (!rel->pgstat_info || !pgstat_track_counts)
/*
@@ -585,12 +573,131 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
*/
return;
- report->blks_fetched =
+ report->blks_fetched +=
rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
- report->blks_hit =
+ report->blks_hit +=
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Set initial values for common heap and index statistics*/
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ memset(report, 0, sizeof(ExtVacReport));
+
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
+ extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
+ extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
+ extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
+ extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
+ extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
+ extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
+ extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
+ extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+
+ extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
+ extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+
+}
+
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacIdxStats)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
+ vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
+ vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
+ vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
+ vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
+ vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
+ vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
+ vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
+ vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
+ vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
+
+ vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -750,11 +857,9 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
LVExtStatCounters extVacCounters;
ExtVacReport extVacReport;
char **indnames = NULL;
- ExtVacReport allzero;
/* Initialize vacuum statistics */
- memset(&allzero, 0, sizeof(ExtVacReport));
- extVacReport = allzero;
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -800,6 +905,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
vac_open_indexes(vacrel->rel, RowExclusiveLock, &vacrel->nindexes,
@@ -1051,23 +1158,6 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
/* Make generic extended vacuum stats report */
extvac_stats_end(rel, &extVacCounters, &extVacReport);
- if(pgstat_track_vacuum_statistics)
- {
- /* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
- extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
- extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
- extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
- extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- }
-
/*
* Report results to the cumulative stats system, too.
*
@@ -1078,13 +1168,34 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(RelationGetRelid(rel),
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
+ vacrel->missed_dead_tuples,
starttime,
&extVacReport);
+
+ }
+ else
+ {
+ pgstat_report_vacuum(RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+ }
+
pgstat_progress_end_command();
if (instrument)
@@ -2781,10 +2892,20 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
* that parallel VACUUM only gets the precheck and this postcheck.
@@ -3205,10 +3326,20 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
estimated_count);
+
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
/* Reset the progress counters */
@@ -3234,6 +3365,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3252,6 +3388,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3260,6 +3397,19 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3284,6 +3434,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3303,12 +3458,25 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 47d27314b55..83d55e78606 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1457,3 +1457,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 2b55d9b7c0e..65de45a4447 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,15 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 23cb62e36a7..f5f75aa4264 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1176,6 +1176,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index ee0385cd809..9ee03509490 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -1016,6 +1016,9 @@ static void
pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
bool accumulate_reltype_specific_info)
{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
dst->total_blks_read += src->total_blks_read;
dst->total_blks_hit += src->total_blks_hit;
dst->total_blks_dirtied += src->total_blks_dirtied;
@@ -1031,20 +1034,35 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
- dst->vm_new_visible_pages += src->vm_new_visible_pages;
- dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->missed_dead_pages += src->missed_dead_pages;
- dst->missed_dead_tuples += src->missed_dead_tuples;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->tuples_deleted += src->tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index a5610199893..482929b75e9 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2372,18 +2372,19 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2402,6 +2403,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 115f0c51cc2..42f4cac5e0e 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1510,7 +1510,7 @@ struct config_bool ConfigureNamesBool[] =
},
{
{"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
- gettext_noop("Collects vacuum statistics for table relations."),
+ gettext_noop("Collects vacuum statistics for relations."),
NULL
},
&pgstat_track_vacuum_statistics,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c04d3880241..8c77ae96100 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12574,4 +12574,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 6d1b2991ce5..fb134f3402e 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -295,6 +296,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -408,4 +429,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 6c88d57aef7..4def2c60d1d 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,11 +111,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -144,18 +152,44 @@ typedef struct ExtVacReport
double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
double total_time; /* total time of a vacuum operation, in msec */
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
- int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
- int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
- int64 missed_dead_pages; /* pages with missed dead tuples */
int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
- int64 index_vacuum_count; /* the number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 87f7e40b4a6..6d960423912 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -34,7 +34,7 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
(1 row)
step s1_commit: COMMIT;
@@ -48,6 +48,6 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
(1 row)
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 10a482e2db4..4e5e5ca54da 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2283,6 +2283,28 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schemaname,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
rel.relname,
stats.relid,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..e00a0fc683c
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+ relid | schemaname | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+------------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+(0 rows)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ee0343c2729..0197830b5cd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -144,4 +144,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..ae146e1d23f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,151 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] v23-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (31.2K, 4-v23-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 2978fa59fc553b1a711731ae83159e4f44241dee Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 4 Feb 2025 17:57:44 +0300
Subject: [PATCH 3/5] Machinery for grabbing an extended vacuum statistics on
databases.
Database vacuum statistics information is the collected general
vacuum statistics indexes and tables owned by the databases, which
they belong to.
In addition to the fact that there are far fewer databases in a system
than relations, vacuum statistics for a database contain fewer statistics
than relations, but they are enough to indicate that something may be
wrong in the system and prompt the administrator to enable extended
monitoring for relations.
So, buffer, wal, statistics of I/O time of read and writen blocks
statistics will be observed because they are collected for both
tables, indexes. In addition, we show the number of errors caught
during operation of the vacuum only for the error level.
wraparound_failsafe_count is a number of times when the vacuum starts
urgent cleanup to prevent wraparound problem which is critical for
the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 17 ++-
src/backend/catalog/system_views.sql | 27 ++++-
src/backend/utils/activity/pgstat.c | 2 +-
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 46 +++++++-
src/backend/utils/adt/pgstatfuncs.c | 100 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 13 ++-
src/include/pgstat.h | 5 +-
.../vacuum-extending-in-repetable-read.spec | 6 ++
src/test/regress/expected/rules.out | 17 +++
.../expected/vacuum_index_statistics.out | 16 +--
...ut => vacuum_tables_and_db_statistics.out} | 87 +++++++++++++--
src/test/regress/parallel_schedule | 2 +-
.../regress/sql/vacuum_index_statistics.sql | 6 +-
...ql => vacuum_tables_and_db_statistics.sql} | 69 +++++++++++-
16 files changed, 381 insertions(+), 35 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (82%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (81%)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 0888be2afea..3d72f74b05e 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -660,7 +660,7 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
@@ -4089,6 +4089,9 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4104,6 +4107,9 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4119,16 +4125,25 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 83d55e78606..0ae31b87989 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1488,4 +1488,29 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.errors AS errors
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index f5f75aa4264..85557736a3a 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-bool pgstat_track_vacuum_statistics = true;
+bool pgstat_track_vacuum_statistics = false;
/* ----------
* state shared with pgstat_*.c
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index b31f20d41bc..65207d30378 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -485,6 +485,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 9ee03509490..1695680ea62 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -205,6 +205,38 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.type = m_type;
+ pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.errors++;
+ dbentry->vacuum_ext.type = m_type;
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
@@ -216,6 +248,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -274,6 +307,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
}
/*
@@ -1030,6 +1073,8 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->blk_write_time += src->blk_write_time;
dst->delay_time += src->delay_time;
dst->total_time += src->total_time;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->errors += src->errors;
if (!accumulate_reltype_specific_info)
return;
@@ -1057,7 +1102,6 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->table.index_vacuum_count += src->table.index_vacuum_count;
dst->table.missed_dead_pages += src->table.missed_dead_pages;
dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
}
else if (dst->type == PGSTAT_EXTVAC_INDEX)
{
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 482929b75e9..a2ece2c36cf 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2383,7 +2383,7 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
@@ -2513,6 +2513,104 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 42f4cac5e0e..a24dec63f3a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1514,7 +1514,7 @@ struct config_bool ConfigureNamesBool[] =
NULL
},
&pgstat_track_vacuum_statistics,
- true,
+ false,
NULL, NULL, NULL
},
{
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8c77ae96100..4e1e29eec7e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12575,12 +12575,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,wraparound_failsafe,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 4def2c60d1d..f8158aa353c 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -154,6 +154,9 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
+ int32 errors;
+ int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+
ExtVacReportType type; /* heap, index, etc. */
/* ----------
@@ -183,7 +186,6 @@ typedef struct ExtVacReport
int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} table;
struct
{
@@ -762,6 +764,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5893d89573d..cfec3159580 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -18,6 +18,9 @@ teardown
}
session s1
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s1_begin_repeatable_read {
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
select count(ival) from test_vacuum_stat_isolation where id>900;
@@ -25,6 +28,9 @@ step s1_begin_repeatable_read {
step s1_commit { COMMIT; }
session s2
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 4e5e5ca54da..f63f25f94d8 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2283,6 +2283,23 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schemaname,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index e00a0fc683c..9e5d33342c9 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -16,8 +16,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -33,12 +37,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -181,3 +180,4 @@ WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
(1 row)
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 82%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index b5ea9c9ab1e..0300e7b6276 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,7 +6,6 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
--------------
@@ -16,8 +15,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -37,12 +40,12 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -225,3 +228,69 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
(1 row)
DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 0197830b5cd..fa2489716cc 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -145,4 +145,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index ae146e1d23f..9b7e645187d 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -14,8 +14,7 @@ SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -33,7 +32,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -149,3 +148,4 @@ FROM pg_stat_vacuum_indexes vt, pg_class c
WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 81%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 5bc34bec64b..ca7dbde9387 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,15 +7,13 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -36,7 +34,13 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -180,4 +184,59 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+DROP TABLE vestat CASCADE;
+
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
\ No newline at end of file
--
2.34.1
[text/x-patch] v23-0004-Vacuum-statistics-have-been-separated-from-regular-r.patch (93.7K, 5-v23-0004-Vacuum-statistics-have-been-separated-from-regular-r.patch)
download | inline diff:
From 52221c338a7a7b842437e3bff0c385192233d3e5 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Jun 2025 22:24:38 +0300
Subject: [PATCH 4/5] Vacuum statistics have been separated from regular
relation and database statistics to reduce memory usage. Dedicated
PGSTAT_KIND_VACUUM_RELATION and PGSTAT_KIND_VACUUM_DB entries were added to
the stats collector to efficiently allocate memory for vacuum-specific
metrics, which require significantly more space per relation.
---
src/backend/access/heap/vacuumlazy.c | 194 ++++++------
src/backend/catalog/heap.c | 1 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/system_views.sql | 178 +++++------
src/backend/commands/dbcommands.c | 1 +
src/backend/commands/vacuumparallel.c | 14 +-
src/backend/utils/activity/Makefile | 1 +
src/backend/utils/activity/pgstat.c | 28 ++
src/backend/utils/activity/pgstat_database.c | 10 +-
src/backend/utils/activity/pgstat_relation.c | 111 +------
src/backend/utils/activity/pgstat_vacuum.c | 215 +++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 285 +++++-------------
src/include/commands/vacuum.h | 2 +-
src/include/pgstat.h | 200 ++++++------
src/include/utils/pgstat_internal.h | 15 +
src/include/utils/pgstat_kind.h | 4 +-
src/test/regress/expected/rules.out | 146 ++++-----
.../expected/vacuum_index_statistics.out | 82 ++---
.../regress/sql/vacuum_index_statistics.sql | 48 +--
19 files changed, 805 insertions(+), 731 deletions(-)
create mode 100644 src/backend/utils/activity/pgstat_vacuum.c
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3d72f74b05e..0c828fb90dd 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -413,7 +413,7 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
- ExtVacReport extVacReportIdx;
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -525,7 +525,7 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters *counters,
- ExtVacReport *report)
+ PgStat_CommonCounts *report)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -586,6 +586,8 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
if(!pgstat_track_vacuum_statistics)
return;
+ memset(&counters->common, 0, sizeof(LVExtStatCounters));
+
/* Set initial values for common heap and index statistics*/
extvac_stats_start(rel, &counters->common);
counters->pages_deleted = counters->tuples_removed = 0;
@@ -603,11 +605,13 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
void
extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report)
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report)
{
- memset(report, 0, sizeof(ExtVacReport));
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ extvac_stats_end(rel, &counters->common, &report->common);
- extvac_stats_end(rel, &counters->common, report);
report->type = PGSTAT_EXTVAC_INDEX;
if (stats != NULL)
@@ -618,7 +622,7 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
*/
/* Fill index-specific extended stats fields */
- report->tuples_deleted =
+ report->common.tuples_deleted =
stats->tuples_removed - counters->tuples_removed;
report->index.pages_deleted =
stats->pages_deleted - counters->pages_deleted;
@@ -641,7 +645,7 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
* procudure.
*/
static void
-accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacStats)
{
if (!pgstat_track_vacuum_statistics)
return;
@@ -653,49 +657,49 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
- extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
- extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
- extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
- extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
- extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
- extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
- extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
- extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
- extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
- extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
}
static void
-accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacIdxStats)
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacIdxStats)
{
if (!pgstat_track_vacuum_statistics)
return;
/* Fill heap-specific extended stats fields */
- vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
- vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
- vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
- vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
- vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
- vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
- vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
- vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
- vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
- vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
-
- vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
}
@@ -855,11 +859,11 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
LVExtStatCounters extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
char **indnames = NULL;
/* Initialize vacuum statistics */
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -905,7 +909,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
- memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+ memset(&vacrel->extVacReportIdx, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
@@ -1156,7 +1161,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
&frozenxid_updated, &minmulti_updated, false);
/* Make generic extended vacuum stats report */
- extvac_stats_end(rel, &extVacCounters, &extVacReport);
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
/*
* Report results to the cumulative stats system, too.
@@ -1168,33 +1173,20 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- if(pgstat_track_vacuum_statistics)
- {
- /* Make generic extended vacuum stats report and
- * fill heap-specific extended stats fields.
- */
- extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
- accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime,
- &extVacReport);
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
- }
- else
- {
- pgstat_report_vacuum(RelationGetRelid(rel),
+ pgstat_report_vacuum_extstats(vacrel->reloid, rel->rd_rel->relisshared, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime,
- NULL);
- }
+ starttime);
pgstat_progress_end_command();
@@ -2893,9 +2885,9 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -2903,7 +2895,7 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
- extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
/*
@@ -3327,9 +3319,9 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -3338,7 +3330,7 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
vacrel->num_index_scans,
estimated_count);
- extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
@@ -3366,7 +3358,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3397,18 +3392,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- if (!ParallelVacuumIsActive(vacrel))
- accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ pgstat_report_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -3435,7 +3425,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3465,17 +3458,13 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_cleanup_one_index(&ivinfo, istat);
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- if (!ParallelVacuumIsActive(vacrel))
- accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
-
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -4075,6 +4064,27 @@ update_relstats_all_indexes(LVRelState *vacrel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+static void
+pgstat_report_vacuum_error()
+{
+ PgStat_VacuumDBCounts *vacuum_dbentry;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ vacuum_dbentry = pgstat_fetch_stat_vacuum_dbentry(MyDatabaseId);
+
+ if(vacuum_dbentry == NULL)
+ return;
+ vacuum_dbentry->errors++;
+}
+
/*
* Error context callback for errors occurring during vacuum. The error
* context messages for index phases should match the messages set in parallel
@@ -4090,7 +4100,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -4108,7 +4118,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -4126,7 +4136,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+ pgstat_report_vacuum_error();
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
@@ -4134,7 +4144,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+ pgstat_report_vacuum_error();
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
@@ -4142,7 +4152,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_TRUNCATE:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index fbaed5359ad..72c8e339c45 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1873,6 +1873,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 739a92bdcc1..e4fa754aab4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2327,6 +2327,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 0ae31b87989..954e2c6cd7b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1417,100 +1417,104 @@ GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
--
CREATE VIEW pg_stat_vacuum_tables AS
-SELECT
- ns.nspname AS schemaname,
- rel.relname AS relname,
- stats.relid as relid,
-
- stats.total_blks_read AS total_blks_read,
- stats.total_blks_hit AS total_blks_hit,
- stats.total_blks_dirtied AS total_blks_dirtied,
- stats.total_blks_written AS total_blks_written,
-
- stats.rel_blks_read AS rel_blks_read,
- stats.rel_blks_hit AS rel_blks_hit,
-
- stats.pages_scanned AS pages_scanned,
- stats.pages_removed AS pages_removed,
- stats.vm_new_frozen_pages AS vm_new_frozen_pages,
- stats.vm_new_visible_pages AS vm_new_visible_pages,
- stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
- stats.missed_dead_pages AS missed_dead_pages,
- stats.tuples_deleted AS tuples_deleted,
- stats.tuples_frozen AS tuples_frozen,
- stats.recently_dead_tuples AS recently_dead_tuples,
- stats.missed_dead_tuples AS missed_dead_tuples,
-
- stats.wraparound_failsafe AS wraparound_failsafe,
- stats.index_vacuum_count AS index_vacuum_count,
- stats.wal_records AS wal_records,
- stats.wal_fpi AS wal_fpi,
- stats.wal_bytes AS wal_bytes,
-
- stats.blk_read_time AS blk_read_time,
- stats.blk_write_time AS blk_write_time,
-
- stats.delay_time AS delay_time,
- stats.total_time AS total_time
-
-FROM pg_class rel
- JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
- LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
-WHERE rel.relkind = 'r';
+ SELECT
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ S.relid as relid,
+
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
+
+ S.pages_scanned AS pages_scanned,
+ S.pages_removed AS pages_removed,
+ S.vm_new_frozen_pages AS vm_new_frozen_pages,
+ S.vm_new_visible_pages AS vm_new_visible_pages,
+ S.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ S.missed_dead_pages AS missed_dead_pages,
+ S.tuples_deleted AS tuples_deleted,
+ S.tuples_frozen AS tuples_frozen,
+ S.recently_dead_tuples AS recently_dead_tuples,
+ S.missed_dead_tuples AS missed_dead_tuples,
+
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.index_vacuum_count AS index_vacuum_count,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+
+ FROM pg_class C JOIN
+ pg_namespace N ON N.oid = C.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(C.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
CREATE VIEW pg_stat_vacuum_indexes AS
-SELECT
- rel.oid as relid,
- ns.nspname AS schemaname,
- rel.relname AS relname,
+ SELECT
+ C.oid AS relid,
+ I.oid AS indexrelid,
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ I.relname AS indexrelname,
- total_blks_read AS total_blks_read,
- total_blks_hit AS total_blks_hit,
- total_blks_dirtied AS total_blks_dirtied,
- total_blks_written AS total_blks_written,
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
- rel_blks_read AS rel_blks_read,
- rel_blks_hit AS rel_blks_hit,
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
- pages_deleted AS pages_deleted,
- tuples_deleted AS tuples_deleted,
+ S.pages_deleted AS pages_deleted,
+ S.tuples_deleted AS tuples_deleted,
- wal_records AS wal_records,
- wal_fpi AS wal_fpi,
- wal_bytes AS wal_bytes,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
- blk_read_time AS blk_read_time,
- blk_write_time AS blk_write_time,
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
- delay_time AS delay_time,
- total_time AS total_time
-FROM
- pg_class rel
- JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
- LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+ FROM
+ pg_class C JOIN
+ pg_index X ON C.oid = X.indrelid JOIN
+ pg_class I ON I.oid = X.indexrelid
+ LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace),
+ LATERAL pg_stat_get_vacuum_indexes(I.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
CREATE VIEW pg_stat_vacuum_database AS
-SELECT
- db.oid as dboid,
- db.datname AS dbname,
-
- stats.db_blks_read AS db_blks_read,
- stats.db_blks_hit AS db_blks_hit,
- stats.total_blks_dirtied AS total_blks_dirtied,
- stats.total_blks_written AS total_blks_written,
-
- stats.wal_records AS wal_records,
- stats.wal_fpi AS wal_fpi,
- stats.wal_bytes AS wal_bytes,
-
- stats.blk_read_time AS blk_read_time,
- stats.blk_write_time AS blk_write_time,
-
- stats.delay_time AS delay_time,
- stats.total_time AS total_time,
- stats.wraparound_failsafe AS wraparound_failsafe,
- stats.errors AS errors
-FROM
- pg_database db,
- LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
+ SELECT
+ D.oid as dboid,
+ D.datname AS dbname,
+
+ S.db_blks_read AS db_blks_read,
+ S.db_blks_hit AS db_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time,
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.errors AS errors
+ FROM
+ pg_database D,
+ LATERAL pg_stat_get_vacuum_database(D.oid) S;
\ No newline at end of file
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 5fbbcdaabb1..c4b910cd928 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1789,6 +1789,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 65de45a4447..3c37d1f07ce 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,7 +869,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -909,14 +909,10 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared,
+ &extVacReport);
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 9c2443e1ecd..183f7514d2d 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 85557736a3a..ca764a3a214 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -478,6 +478,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 65207d30378..80e6c7c229a 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
@@ -485,7 +494,6 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
- memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 1695680ea62..acc8f0b8a52 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
@@ -205,50 +203,17 @@ pgstat_drop_relation(Relation rel)
}
}
-/* ---------
- * pgstat_report_vacuum_error() -
- *
- * Tell the collector about an (auto)vacuum interruption.
- * ---------
- */
-void
-pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
-{
- PgStat_EntryRef *entry_ref;
- PgStatShared_Relation *shtabentry;
- PgStat_StatTabEntry *tabentry;
- Oid dboid = MyDatabaseId;
- PgStat_StatDBEntry *dbentry; /* pending database entry */
-
- if (!pgstat_track_counts)
- return;
-
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
- dboid, tableoid, false);
-
- shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
- tabentry = &shtabentry->stats;
-
- tabentry->vacuum_ext.type = m_type;
- pgstat_unlock_entry(entry_ref);
-
- dbentry = pgstat_prep_database_pending(dboid);
- dbentry->vacuum_ext.errors++;
- dbentry->vacuum_ext.type = m_type;
-}
-
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params)
+ TimestampTz starttime)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
- PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -270,8 +235,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
- pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
-
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -307,16 +270,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
-
- if (dboid != InvalidOid)
- {
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dboid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
-
- pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
- pgstat_unlock_entry(entry_ref);
- }
}
/*
@@ -951,6 +904,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
@@ -1053,60 +1012,4 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_updated = trans->updated_pre_truncdrop;
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
-}
-
-static void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info)
-{
- if(!pgstat_track_vacuum_statistics)
- return;
-
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->total_time += src->total_time;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->errors += src->errors;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_TABLE)
- {
- dst->table.pages_scanned += src->table.pages_scanned;
- dst->table.pages_removed += src->table.pages_removed;
- dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
- dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
- dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->table.tuples_frozen += src->table.tuples_frozen;
- dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
- dst->table.index_vacuum_count += src->table.index_vacuum_count;
- dst->table.missed_dead_pages += src->table.missed_dead_pages;
- dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->tuples_deleted += src->tuples_deleted;
- }
- }
}
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat_vacuum.c b/src/backend/utils/activity/pgstat_vacuum.c
new file mode 100644
index 00000000000..e11f19e46b2
--- /dev/null
+++ b/src/backend/utils/activity/pgstat_vacuum.c
@@ -0,0 +1,215 @@
+#include "postgres.h"
+
+#include "pgstat.h"
+#include "utils/pgstat_internal.h"
+#include "utils/memutils.h"
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+bool pgstat_track_vacuum_statistics_for_relations = false;
+
+#define ACCUMULATE_FIELD(field) dst->field += src->field;
+
+#define ACCUMULATE_SUBFIELD(substruct, field) \
+ (dst->substruct.field += src->substruct.field)
+
+static void
+pgstat_accumulate_common(PgStat_CommonCounts *dst, const PgStat_CommonCounts *src)
+{
+ ACCUMULATE_FIELD(total_blks_read);
+ ACCUMULATE_FIELD(total_blks_hit);
+ ACCUMULATE_FIELD(total_blks_dirtied);
+ ACCUMULATE_FIELD(total_blks_written);
+
+ ACCUMULATE_FIELD(blks_fetched);
+ ACCUMULATE_FIELD(blks_hit);
+
+ ACCUMULATE_FIELD(wal_records);
+ ACCUMULATE_FIELD(wal_fpi);
+ ACCUMULATE_FIELD(wal_bytes);
+
+ ACCUMULATE_FIELD(blk_read_time);
+ ACCUMULATE_FIELD(blk_write_time);
+ ACCUMULATE_FIELD(delay_time);
+ ACCUMULATE_FIELD(total_time);
+
+ ACCUMULATE_FIELD(tuples_deleted);
+ ACCUMULATE_FIELD(wraparound_failsafe_count);
+}
+
+static void
+pgstat_accumulate_extvac_stats_relations(PgStat_VacuumRelationCounts *dst, PgStat_VacuumRelationCounts *src)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB && src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ ACCUMULATE_SUBFIELD(common, blks_fetched);
+ ACCUMULATE_SUBFIELD(common, blks_hit);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(table, pages_scanned);
+ ACCUMULATE_SUBFIELD(table, pages_removed);
+ ACCUMULATE_SUBFIELD(table, vm_new_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, tuples_frozen);
+ ACCUMULATE_SUBFIELD(table, recently_dead_tuples);
+ ACCUMULATE_SUBFIELD(table, index_vacuum_count);
+ ACCUMULATE_SUBFIELD(table, missed_dead_pages);
+ ACCUMULATE_SUBFIELD(table, missed_dead_tuples);
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(index, pages_deleted);
+ }
+}
+
+static void
+pgstat_accumulate_extvac_stats_db(PgStat_VacuumDBCounts *dst, PgStat_VacuumDBCounts *src)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+ dst->errors += src->errors;
+}
+
+/*
+ * Report that the table was just vacuumed and flush statistics.
+ */
+void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_VacuumRelation *shtabentry;
+ PgStatShared_VacuumDB *shdbentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_RELATION,
+ dboid, tableoid, false);
+ shtabentry = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+ pgstat_accumulate_extvac_stats_relations(&shtabentry->stats, params);
+
+ pgstat_unlock_entry(entry_ref);
+
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_DB,
+ dboid, InvalidOid, false);
+
+ shdbentry = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ pgstat_accumulate_common(&shdbentry->stats.common, ¶ms->common);
+
+ pgstat_unlock_entry(entry_ref);
+}
+
+/*
+ * Flush out pending stats for the entry
+ *
+ * If nowait is true, this function returns false if lock could not
+ * immediately acquired, otherwise true is returned.
+ */
+bool
+pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumRelation *shtabstats;
+ PgStat_RelationVacuumPending *pendingent; /* table entry of shared stats */
+
+ pendingent = (PgStat_RelationVacuumPending *) entry_ref->pending;
+ shtabstats = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts.
+ */
+ if (pg_memory_is_all_zeros(&pendingent,
+ sizeof(struct PgStat_RelationVacuumPending)))
+ return true;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ {
+ return false;
+ }
+
+ pgstat_accumulate_extvac_stats_relations(&(shtabstats->stats), &(pendingent->counts));
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the vacuum collected statistics for one relation or NULL.
+ */
+PgStat_VacuumRelationCounts *
+pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid)
+{
+ return (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_RELATION, dbid, relid);
+}
+
+PgStat_VacuumDBCounts *
+pgstat_fetch_stat_vacuum_dbentry(Oid dbid)
+{
+ return (PgStat_VacuumDBCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_DB, dbid, InvalidOid);
+}
+
+bool
+pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumDB *sharedent;
+ PgStat_VacuumDBCounts *pendingent;
+
+ pendingent = (PgStat_VacuumDBCounts *) entry_ref->pending;
+ sharedent = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ /* The entry was successfully flushed, add the same to database stats */
+ pgstat_accumulate_extvac_stats_db(&(sharedent->stats), pendingent);
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Find or create a local PgStat_VacuumDBCounts entry for dboid.
+ */
+PgStat_VacuumDBCounts *
+pgstat_prep_vacuum_database_pending(Oid dboid)
+{
+ PgStat_EntryRef *entry_ref;
+
+ /*
+ * This should not report stats on database objects before having
+ * connected to a database.
+ */
+ Assert(!OidIsValid(dboid) || OidIsValid(MyDatabaseId));
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_VACUUM_DB, dboid, InvalidOid,
+ NULL);
+
+ if(entry_ref == NULL)
+ return NULL;
+
+ return entry_ref->pending;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index a2ece2c36cf..8603d4dd576 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2265,7 +2265,6 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
-
/*
* Get the vacuum statistics for the heap tables.
*/
@@ -2275,102 +2274,42 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
-
- Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- tabentry = pgstat_fetch_stat_tabentry(relid);
-
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = pending;
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
@@ -2378,28 +2317,28 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
@@ -2416,100 +2355,60 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
-
- Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
-
- tabentry = pgstat_fetch_stat_tabentry(relid);
-
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = pending;
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
@@ -2523,90 +2422,52 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
Oid dbid = PG_GETARG_OID(0);
- PgStat_StatDBEntry *dbentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumDBCounts *extvacuum;
+ PgStat_VacuumDBCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
-
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
+ PgStat_VacuumDBCounts allzero;
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
- INT4OID, -1, 0);
+ pending = pgstat_fetch_stat_vacuum_dbentry(dbid);
- Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
-
- dbentry = pgstat_fetch_stat_dbentry(dbid);
-
- if (dbentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(dbentry->vacuum_ext);
- }
-
- i = 0;
+ extvacuum = pending;
values[i++] = ObjectIdGetDatum(dbid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
values[i++] = Int32GetDatum(extvacuum->errors);
Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index fb134f3402e..f895151ca09 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -432,5 +432,5 @@ extern double anl_get_next_S(double t, int n, double *stateptr);
extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx *counters);
extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report);
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f8158aa353c..3a8a43b3813 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -116,46 +116,100 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_TABLE = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
+ * PgStat_TableCounts The actual per-table counts kept by a backend
*
- * ExtVacReport
+ * This struct should contain only actual event counters, because we make use
+ * of pg_memory_is_all_zeros() to detect whether there are any stats updates
+ * to apply.
*
- * Additional statistics of vacuum processing over a relation.
- * pages_removed is the amount by which the physically shrank,
- * if any (ie the change in its total size on disk)
- * pages_deleted refer to free space within the index file
+ * It is a component of PgStat_TableStatus (within-backend state).
+ *
+ * Note: for a table, tuples_returned is the number of tuples successfully
+ * fetched by heap_getnext, while tuples_fetched is the number of tuples
+ * successfully fetched by heap_fetch under the control of bitmap indexscans.
+ * For an index, tuples_returned is the number of index entries returned by
+ * the index AM, while tuples_fetched is the number of tuples successfully
+ * fetched by heap_fetch under the control of simple indexscans for this index.
+ *
+ * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
+ * actions, regardless of whether the transaction committed. delta_live_tuples,
+ * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
+ * Note that delta_live_tuples and delta_dead_tuples can be negative!
* ----------
*/
-typedef struct ExtVacReport
+typedef struct PgStat_TableCounts
{
- /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
- int64 total_blks_read;
- int64 total_blks_hit;
- int64 total_blks_dirtied;
- int64 total_blks_written;
+ PgStat_Counter numscans;
- /* blocks missed and hit for just the heap during a vacuum of specific relation */
- int64 blks_fetched;
- int64 blks_hit;
+ PgStat_Counter tuples_returned;
+ PgStat_Counter tuples_fetched;
- /* Vacuum WAL usage stats */
- int64 wal_records; /* wal usage: number of WAL records */
- int64 wal_fpi; /* wal usage: number of WAL full page images produced */
- uint64 wal_bytes; /* wal usage: size of WAL records produced */
+ PgStat_Counter tuples_inserted;
+ PgStat_Counter tuples_updated;
+ PgStat_Counter tuples_deleted;
+ PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_newpage_updated;
+ bool truncdropped;
+
+ PgStat_Counter delta_live_tuples;
+ PgStat_Counter delta_dead_tuples;
+ PgStat_Counter changed_tuples;
+
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
- /* Time stats. */
- double blk_read_time; /* time spent reading pages, in msec */
- double blk_write_time; /* time spent writing pages, in msec */
- double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
- double total_time; /* total time of a vacuum operation, in msec */
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+} PgStat_TableCounts;
- int64 tuples_deleted; /* tuples deleted by vacuum */
+typedef struct PgStat_CommonCounts
+{
+ /* blocks */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* heap blocks */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* WAL */
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+
+ /* Time */
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+
+ /* tuples */
+ int64 tuples_deleted;
+
+ /* failsafe */
+ int32 wraparound_failsafe_count;
+} PgStat_CommonCounts;
- int32 errors;
- int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
ExtVacReportType type; /* heap, index, etc. */
@@ -174,16 +228,16 @@ typedef struct ExtVacReport
{
struct
{
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 pages_scanned; /* heap pages examined (not skipped by VM) */
int64 pages_removed; /* heap pages removed by vacuum "truncation" */
int64 pages_frozen; /* pages marked in VM as frozen */
int64 pages_all_visible; /* pages marked in VM as all-visible */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
} table;
@@ -192,61 +246,21 @@ typedef struct ExtVacReport
int64 pages_deleted; /* number of pages deleted by vacuum */
} index;
} /* per_type_stats */;
-} ExtVacReport;
+} PgStat_VacuumRelationCounts;
-/* ----------
- * PgStat_TableCounts The actual per-table counts kept by a backend
- *
- * This struct should contain only actual event counters, because we make use
- * of pg_memory_is_all_zeros() to detect whether there are any stats updates
- * to apply.
- *
- * It is a component of PgStat_TableStatus (within-backend state).
- *
- * Note: for a table, tuples_returned is the number of tuples successfully
- * fetched by heap_getnext, while tuples_fetched is the number of tuples
- * successfully fetched by heap_fetch under the control of bitmap indexscans.
- * For an index, tuples_returned is the number of index entries returned by
- * the index AM, while tuples_fetched is the number of tuples successfully
- * fetched by heap_fetch under the control of simple indexscans for this index.
- *
- * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
- * actions, regardless of whether the transaction committed. delta_live_tuples,
- * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
- * Note that delta_live_tuples and delta_dead_tuples can be negative!
- * ----------
- */
-typedef struct PgStat_TableCounts
+typedef struct PgStat_VacuumRelationStatus
{
- PgStat_Counter numscans;
-
- PgStat_Counter tuples_returned;
- PgStat_Counter tuples_fetched;
-
- PgStat_Counter tuples_inserted;
- PgStat_Counter tuples_updated;
- PgStat_Counter tuples_deleted;
- PgStat_Counter tuples_hot_updated;
- PgStat_Counter tuples_newpage_updated;
- bool truncdropped;
-
- PgStat_Counter delta_live_tuples;
- PgStat_Counter delta_dead_tuples;
- PgStat_Counter changed_tuples;
-
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-
- PgStat_Counter rev_all_visible_pages;
- PgStat_Counter rev_all_frozen_pages;
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
- /*
- * Additional cumulative stat on vacuum operations.
- * Use an expensive structure as an abstraction for different types of
- * relations.
- */
- ExtVacReport vacuum_ext;
-} PgStat_TableCounts;
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ PgStat_CommonCounts common;
+ int32 errors;
+} PgStat_VacuumDBCounts;
/* ----------
* PgStat_TableStatus Per-table status within a backend
@@ -272,6 +286,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -468,8 +488,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
-
- ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -551,8 +569,6 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter rev_all_visible_pages;
PgStat_Counter rev_all_frozen_pages;
-
- ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -760,11 +776,10 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params);
+ TimestampTz starttime);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
-extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -895,6 +910,15 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts *pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts *pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid);
+PgStat_VacuumDBCounts *pgstat_fetch_stat_vacuum_dbentry(Oid dbid);
/*
* Functions in pgstat_wal.c
*/
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index d5557e6e998..140adbcdbd6 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -439,6 +439,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -607,6 +619,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index f44169fd5a3..454661f9d6a 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f63f25f94d8..767175e2a66 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2283,77 +2283,81 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
-pg_stat_vacuum_database| SELECT db.oid AS dboid,
- db.datname AS dbname,
- stats.db_blks_read,
- stats.db_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time,
- stats.wraparound_failsafe,
- stats.errors
- FROM pg_database db,
- LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
-pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
- ns.nspname AS schemaname,
- rel.relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_deleted,
- stats.tuples_deleted,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time
- FROM (pg_class rel
- JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
- LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
- WHERE (rel.relkind = 'i'::"char");
-pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
- rel.relname,
- stats.relid,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_scanned,
- stats.pages_removed,
- stats.vm_new_frozen_pages,
- stats.vm_new_visible_pages,
- stats.vm_new_visible_frozen_pages,
- stats.missed_dead_pages,
- stats.tuples_deleted,
- stats.tuples_frozen,
- stats.recently_dead_tuples,
- stats.missed_dead_tuples,
- stats.wraparound_failsafe,
- stats.index_vacuum_count,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time
- FROM (pg_class rel
- JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
- LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
- WHERE (rel.relkind = 'r'::"char");
+pg_stat_vacuum_database| SELECT d.oid AS dboid,
+ d.datname AS dbname,
+ s.db_blks_read,
+ s.db_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time,
+ s.wraparound_failsafe,
+ s.errors
+ FROM pg_database d,
+ LATERAL pg_stat_get_vacuum_database(d.oid) s(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
+pg_stat_vacuum_indexes| SELECT c.oid AS relid,
+ i.oid AS indexrelid,
+ n.nspname AS schemaname,
+ c.relname,
+ i.relname AS indexrelname,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_deleted,
+ s.tuples_deleted,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (((pg_class c
+ JOIN pg_index x ON ((c.oid = x.indrelid)))
+ JOIN pg_class i ON ((i.oid = x.indexrelid)))
+ LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(i.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
+pg_stat_vacuum_tables| SELECT n.nspname AS schemaname,
+ c.relname,
+ s.relid,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_scanned,
+ s.pages_removed,
+ s.vm_new_frozen_pages,
+ s.vm_new_visible_pages,
+ s.vm_new_visible_frozen_pages,
+ s.missed_dead_pages,
+ s.tuples_deleted,
+ s.tuples_frozen,
+ s.recently_dead_tuples,
+ s.missed_dead_tuples,
+ s.wraparound_failsafe,
+ s.index_vacuum_count,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (pg_class c
+ JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(c.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index 9e5d33342c9..4654a536ad6 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -30,9 +30,9 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- Must be empty.
SELECT *
FROM pg_stat_vacuum_indexes vt
-WHERE vt.relname = 'vestat';
- relid | schemaname | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
--------+------------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+WHERE vt.indexrelname = 'vestat';
+ relid | indexrelid | schemaname | relname | indexrelname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+------------+------------+---------+--------------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
(0 rows)
RESET track_vacuum_statistics;
@@ -55,12 +55,12 @@ ANALYZE vestat;
SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
DELETE FROM vestat WHERE x % 2 = 0;
-- Before the first vacuum execution extended stats view is empty.
-SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+SELECT vt.indexrelname,relpages,pages_deleted,tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | 30 | 0 | 0
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
(1 row)
SELECT relpages AS irp
@@ -72,22 +72,22 @@ CHECKPOINT;
-- The table and index extended vacuum statistics should show us that
-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
-- because of not full table have cleaned up
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | t | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
(1 row)
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- Look into WAL records deltas.
SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey';
diwr | diwb
------+------
t | t
@@ -98,20 +98,20 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
-- it is necessary to check the wal statistics
CHECKPOINT;
-- pages_removed must be increased
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | t | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
(1 row)
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL advance should be detected.
SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
diwr | diwb
@@ -120,7 +120,7 @@ SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
(1 row)
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
DELETE FROM vestat WHERE x % 2 = 0;
-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
@@ -130,7 +130,7 @@ VACUUM FULL vestat;
CHECKPOINT;
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL and other statistics advance should not be detected.
SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
diwr | ifpi | diwb
@@ -138,19 +138,19 @@ SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
t | t | t
(1 row)
-SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | t | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
(1 row)
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
DELETE FROM vestat;
TRUNCATE vestat;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
@@ -158,7 +158,7 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
CHECKPOINT;
-- Store WAL advances into variables after removing all tuples from the table
SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
--There are nothing changed
SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
diwr | ifpi | diwb
@@ -171,12 +171,12 @@ SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
-- in vacuum extended statistics.
-- The pages_frozen, pages_scanned values shouldn't be changed
--
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | f | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
(1 row)
DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index 9b7e645187d..57e5420b9b6 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -27,7 +27,7 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- Must be empty.
SELECT *
FROM pg_stat_vacuum_indexes vt
-WHERE vt.relname = 'vestat';
+WHERE vt.indexrelname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
@@ -49,9 +49,9 @@ SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
DELETE FROM vestat WHERE x % 2 = 0;
-- Before the first vacuum execution extended stats view is empty.
-SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+SELECT vt.indexrelname,relpages,pages_deleted,tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
SELECT relpages AS irp
FROM pg_class c
WHERE relname = 'vestat_pkey' \gset
@@ -63,19 +63,19 @@ CHECKPOINT;
-- The table and index extended vacuum statistics should show us that
-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
-- because of not full table have cleaned up
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- Look into WAL records deltas.
SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey';
DELETE FROM vestat;;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
@@ -83,22 +83,22 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
CHECKPOINT;
-- pages_removed must be increased
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL advance should be detected.
SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
DELETE FROM vestat WHERE x % 2 = 0;
@@ -110,20 +110,20 @@ CHECKPOINT;
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL and other statistics advance should not be detected.
SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
-SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
DELETE FROM vestat;
TRUNCATE vestat;
@@ -133,7 +133,7 @@ CHECKPOINT;
-- Store WAL advances into variables after removing all tuples from the table
SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
--There are nothing changed
SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
@@ -143,9 +143,9 @@ SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
-- in vacuum extended statistics.
-- The pages_frozen, pages_scanned values shouldn't be changed
--
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
DROP TABLE vestat;
RESET track_vacuum_statistics;
--
2.34.1
[text/x-patch] v23-0005-Add-documentation-about-the-system-views-that-are-us.patch (24.5K, 6-v23-0005-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 5f07a5bbe804b858e7e15e8bb64c96ccc787b184 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 5/5] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 755 +++++++++++++++++++++++++++++++++
1 file changed, 755 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index b58c52ea50f..7e5acd7c52e 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5474,4 +5474,759 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.34.1
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-09-01 19:13 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2025-09-01 19:13 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
I've rebased the patches to the current HEAD.
--
Regards,
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] v24-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (71.3K, 2-v24-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From bf8e64e8996f718465b0ee8093d2e5efed6064a8 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 1 Sep 2025 21:40:24 +0300
Subject: [PATCH 1/5] Machinery for grabbing an extended vacuum statistics on
table relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen is a number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible is a number of pages that are marked as all-visible in vm during
vacuum.
wraparound_failsafe_count is a number of times when the vacuum starts urgent cleanup
to prevent wraparound problem which is critical for the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 150 +++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 52 +++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 46 +++-
src/backend/utils/adt/pgstatfuncs.c | 147 ++++++++++++
src/backend/utils/error/elog.c | 13 +
src/backend/utils/misc/guc_tables.c | 10 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 80 +++++-
src/include/utils/elog.h | 1 +
.../vacuum-extending-in-repetable-read.out | 53 ++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 53 ++++
src/test/regress/expected/rules.out | 44 +++-
.../expected/vacuum_tables_statistics.out | 227 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 183 ++++++++++++++
22 files changed, 1096 insertions(+), 16 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 932701d8420..980101e9b97 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -289,6 +289,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -407,6 +408,8 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} LVRelState;
@@ -418,6 +421,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -474,6 +489,106 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
@@ -632,6 +747,13 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
char **indnames = NULL;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -651,7 +773,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -668,6 +790,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -776,6 +899,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->aggressive = vacuum_get_cutoffs(rel, params, &vacrel->cutoffs);
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ vacrel->wraparound_failsafe_count = 0;
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
@@ -924,6 +1048,26 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ }
+
/*
* Report results to the cumulative stats system, too.
*
@@ -939,7 +1083,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime);
+ starttime,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -2961,6 +3106,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[2] = {0, 0};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count ++;
/*
* Abandon use of a buffer access strategy to allow use of all of
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 953ad4a4843..a21e77cd551 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1b3c5a55882..6480c3fc92a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -716,7 +716,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_vacuum_time(C.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1420,3 +1422,51 @@ REVOKE ALL ON pg_aios FROM PUBLIC;
GRANT SELECT ON pg_aios TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_aios() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.vm_new_frozen_pages AS vm_new_frozen_pages,
+ stats.vm_new_visible_pages AS vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ stats.missed_dead_pages AS missed_dead_pages,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.recently_dead_tuples AS recently_dead_tuples,
+ stats.missed_dead_tuples AS missed_dead_tuples,
+
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 733ef40ae7c..d8776ff1901 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -116,6 +116,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2533,6 +2536,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 0feea1d30ec..2b55d9b7c0e 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1054,6 +1054,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index ffb5b8cce34..a19d4a770b8 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = true;
/* ----------
* state shared with pgstat_*.c
@@ -265,7 +265,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -883,7 +882,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -952,7 +950,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1068,7 +1066,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1119,7 +1117,7 @@ pgstat_prep_snapshot(void)
}
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 69df741cbf6..e023926ff05 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -209,7 +211,7 @@ pgstat_drop_relation(Relation rel)
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime)
+ TimestampTz starttime, ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -235,6 +237,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -881,6 +885,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -1004,3 +1011,40 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
+ dst->vm_new_visible_pages += src->vm_new_visible_pages;
+ dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->missed_dead_pages += src->missed_dead_pages;
+ dst->missed_dead_tuples += src->missed_dead_tuples;
+
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..ee461ea378d 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2260,3 +2266,144 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index b7b9692f8c8..f0ecf86e514 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1627,6 +1627,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f137129209f..5af593aa1a7 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1568,6 +1568,16 @@ struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
+ {
+ {"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
+ gettext_noop("Collects vacuum statistics for table relations."),
+ NULL
+ },
+ &pgstat_track_vacuum_statistics,
+ true,
+ NULL, NULL, NULL
+ },
+
{
{"track_wal_io_timing", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects timing statistics for WAL I/O activity."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index a9d8293474a..05713b4b65c 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -661,6 +661,7 @@
#track_wal_io_timing = off
#track_functions = none # none, pl, all
#stats_fetch_consistency = cache # cache, none, snapshot
+#track_vacuum_statistics = off
# - Monitoring -
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 118d6da1ace..687d51c2a6a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12576,4 +12576,22 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int4,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 14eeccbd718..4d05e1a0fac 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -327,6 +327,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 202bd2d5ace..4fd114e9a5f 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,6 +111,53 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* the number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -153,6 +200,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -211,7 +268,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB7
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB8
typedef struct PgStat_ArchiverStats
{
@@ -375,6 +432,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -453,6 +512,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autovacuum_time;
PgStat_Counter total_analyze_time;
PgStat_Counter total_autoanalyze_time;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -660,7 +724,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime);
+ TimestampTz starttime, ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -711,6 +775,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -799,6 +874,7 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
/*
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 675f4f5f469..356dadd6b0a 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrcode(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..87f7e40b4a6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 9f1e997d81b..815dcee23b2 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -97,6 +97,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5893d89573d
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,53 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..349e7deba01 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1833,7 +1833,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_vacuum_time(c.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2232,7 +2234,9 @@ pg_stat_sys_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2284,9 +2288,43 @@ pg_stat_user_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.missed_dead_pages,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.missed_dead_tuples,
+ stats.wraparound_failsafe,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..b5ea9c9ab1e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,227 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+ relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_scanned | pages_removed | vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | missed_dead_pages | tuples_deleted | tuples_frozen | recently_dead_tuples | missed_dead_tuples | index_vacuum_count | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+---------------+---------------------+----------------------+-----------------------------+-------------------+----------------+---------------+----------------------+--------------------+--------------------+-------------+---------+-----------+---------------+----------------+------------+------------
+ vestat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
+(1 row)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
+---------------------+----------------------+----------------------+-----------------------+-----------------------------
+ 0 | 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | f | f
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index fbffc67ae60..cd779ab8eca 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,3 +140,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..5bc34bec64b
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v24-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (55.1K, 3-v24-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From c420c88f6c6aa326de47954cf552389efa7c3c37 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 1 Sep 2025 21:42:25 +0300
Subject: [PATCH 2/5] Machinery for grabbing an extended vacuum statistics on
index relations.
They are gathered separatelly from table statistics.
As for tables, we gather vacuum shared buffers statistics for index relations like
value of total_blks_hit, total_blks_read, total_blks_dirtied, wal statistics, io time
during flushing buffer pages to disk, delay and total time.
Due to the fact that such statistics are common as for tables, as for indexes we
set them in the union ExtVacReport structure. We only added some determination 'type'
field to highlight what kind belong to these statistics: PGSTAT_EXTVAC_TABLE or
PGSTAT_EXTVAC_INDEX. Generally, PGSTAT_EXTVAC_INVALID type leads to wrong code process.
Some statistics belong only one type of both tables or indexes. So, we added substructures
sych table and index inside ExtVacReport structure.
Therefore, we gather only for tables such statistics like number of scanned, removed pages,
their charecteristics according VM (all-visible and frozen). In addition, for tables we
gather number frozen, deleted and recently dead tuples and how many times vacuum processed
indexes for tables.
Controversally for indexes we gather number of deleted pages and deleted tuples only.
As for tables, deleted pages and deleted tuples reflect the overall performance of the vacuum
for the index relationship.
Since the vacuum cleans up references to tuple indexes before cleaning up table tuples,
which adds some complexity to the vacuum process, namely the vacuum switches from cleaning up
a table to its indexes and back during its operation, we need to save the vacuum statistics
collected for the heap before it starts cleaning up the indexes.
That's why it's necessary to track the vacuum statistics for the heap several times during
the vacuum procedure. To avoid sending the statistics to the Cumulative Statistics System
several times, we save these statistics in the LVRelState structure and only after vacuum
finishes cleaning up the heap, it sends them to the Cumulative Statistics System.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 268 ++++++++++++++----
src/backend/catalog/system_views.sql | 32 +++
src/backend/commands/vacuumparallel.c | 14 +
src/backend/utils/activity/pgstat.c | 4 +
src/backend/utils/activity/pgstat_relation.c | 48 +++-
src/backend/utils/adt/pgstatfuncs.c | 133 ++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 25 ++
src/include/pgstat.h | 58 +++-
.../vacuum-extending-in-repetable-read.out | 4 +-
src/test/regress/expected/rules.out | 22 ++
.../expected/vacuum_index_statistics.out | 183 ++++++++++++
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 151 ++++++++++
15 files changed, 864 insertions(+), 90 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 980101e9b97..e206aeecc9c 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -290,6 +290,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -410,6 +411,8 @@ typedef struct LVRelState
BlockNumber eager_scan_remaining_fails;
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReport extVacReportIdx;
} LVRelState;
@@ -421,19 +424,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-/*
- * Counters and usage data for extended stats tracking.
- */
-typedef struct LVExtStatCounters
-{
- TimestampTz starttime;
- WalUsage walusage;
- BufferUsage bufusage;
- double VacuumDelayTime;
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-} LVExtStatCounters;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static void heap_vacuum_eager_scan_setup(LVRelState *vacrel,
@@ -555,27 +545,25 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
endtime = GetCurrentTimestamp();
TimestampDifference(counters->starttime, endtime, &secs, &usecs);
- memset(report, 0, sizeof(ExtVacReport));
-
/*
* Fill additional statistics on a vacuum processing operation.
*/
- report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
- report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
- report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
- report->total_blks_written = bufusage.shared_blks_written;
+ report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written += bufusage.shared_blks_written;
- report->wal_records = walusage.wal_records;
- report->wal_fpi = walusage.wal_fpi;
- report->wal_bytes = walusage.wal_bytes;
+ report->wal_records += walusage.wal_records;
+ report->wal_fpi += walusage.wal_fpi;
+ report->wal_bytes += walusage.wal_bytes;
- report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
- report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
- report->total_time = secs * 1000. + usecs / 1000.;
+ report->total_time += secs * 1000. + usecs / 1000.;
if (!rel->pgstat_info || !pgstat_track_counts)
/*
@@ -584,12 +572,131 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
*/
return;
- report->blks_fetched =
+ report->blks_fetched +=
rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
- report->blks_hit =
+ report->blks_hit +=
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Set initial values for common heap and index statistics*/
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ memset(report, 0, sizeof(ExtVacReport));
+
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
+ extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
+ extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
+ extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
+ extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
+ extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
+ extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
+ extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
+ extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+
+ extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
+ extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+
+}
+
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacIdxStats)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
+ vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
+ vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
+ vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
+ vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
+ vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
+ vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
+ vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
+ vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
+ vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
+
+ vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -752,7 +859,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ExtVacReport allzero;
/* Initialize vacuum statistics */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
extVacReport = allzero;
verbose = (params.options & VACOPT_VERBOSE) != 0;
@@ -799,6 +906,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
vac_open_indexes(vacrel->rel, RowExclusiveLock, &vacrel->nindexes,
@@ -1051,23 +1160,6 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Make generic extended vacuum stats report */
extvac_stats_end(rel, &extVacCounters, &extVacReport);
- if(pgstat_track_vacuum_statistics)
- {
- /* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
- extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
- extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
- extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
- extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- }
-
/*
* Report results to the cumulative stats system, too.
*
@@ -1078,13 +1170,34 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(RelationGetRelid(rel),
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
+ vacrel->missed_dead_tuples,
starttime,
&extVacReport);
+
+ }
+ else
+ {
+ pgstat_report_vacuum(RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+ }
+
pgstat_progress_end_command();
if (instrument)
@@ -2776,10 +2889,20 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
* that parallel VACUUM only gets the precheck and this postcheck.
@@ -3189,10 +3312,20 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
estimated_count);
+
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
/* Reset the progress counters */
@@ -3218,6 +3351,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3236,6 +3374,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3244,6 +3383,19 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3268,6 +3420,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3287,12 +3444,25 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 6480c3fc92a..eb63be1833a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1470,3 +1470,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 2b55d9b7c0e..65de45a4447 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,15 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index a19d4a770b8..2b7058e5c01 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1162,6 +1162,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index e023926ff05..c6194584b35 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -1016,6 +1016,9 @@ static void
pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
bool accumulate_reltype_specific_info)
{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
dst->total_blks_read += src->total_blks_read;
dst->total_blks_hit += src->total_blks_hit;
dst->total_blks_dirtied += src->total_blks_dirtied;
@@ -1031,20 +1034,35 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
- dst->vm_new_visible_pages += src->vm_new_visible_pages;
- dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->missed_dead_pages += src->missed_dead_pages;
- dst->missed_dead_tuples += src->missed_dead_tuples;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->tuples_deleted += src->tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index ee461ea378d..defe1990e11 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2374,18 +2374,19 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2404,6 +2405,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 5af593aa1a7..ac6427cf1e9 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1570,7 +1570,7 @@ struct config_bool ConfigureNamesBool[] =
{
{"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
- gettext_noop("Collects vacuum statistics for table relations."),
+ gettext_noop("Collects vacuum statistics for relations."),
NULL
},
&pgstat_track_vacuum_statistics,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 687d51c2a6a..2fbd001178e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12594,4 +12594,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 4d05e1a0fac..dcc542750b8 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -295,6 +296,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -408,4 +429,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 4fd114e9a5f..aec25ad2262 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,11 +111,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -144,18 +152,44 @@ typedef struct ExtVacReport
double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
double total_time; /* total time of a vacuum operation, in msec */
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
- int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
- int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
- int64 missed_dead_pages; /* pages with missed dead tuples */
int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
- int64 index_vacuum_count; /* the number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 87f7e40b4a6..6d960423912 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -34,7 +34,7 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
(1 row)
step s1_commit: COMMIT;
@@ -48,6 +48,6 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
(1 row)
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 349e7deba01..fdd5341bdfc 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2293,6 +2293,28 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schemaname,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
rel.relname,
stats.relid,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..e00a0fc683c
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+ relid | schemaname | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+------------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+(0 rows)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index cd779ab8eca..4eb03353104 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -144,4 +144,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..ae146e1d23f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,151 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] v24-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (31.2K, 4-v24-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 789ed32efc0ac06792d4e6cbe6759af7ad8aeb3c Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 1 Sep 2025 21:43:33 +0300
Subject: [PATCH 3/5] Machinery for grabbing an extended vacuum statistics on
databases.
Database vacuum statistics information is the collected general
vacuum statistics indexes and tables owned by the databases, which
they belong to.
In addition to the fact that there are far fewer databases in a system
than relations, vacuum statistics for a database contain fewer statistics
than relations, but they are enough to indicate that something may be
wrong in the system and prompt the administrator to enable extended
monitoring for relations.
So, buffer, wal, statistics of I/O time of read and writen blocks
statistics will be observed because they are collected for both
tables, indexes. In addition, we show the number of errors caught
during operation of the vacuum only for the error level.
wraparound_failsafe_count is a number of times when the vacuum starts
urgent cleanup to prevent wraparound problem which is critical for
the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 17 ++-
src/backend/catalog/system_views.sql | 27 ++++-
src/backend/utils/activity/pgstat.c | 2 +-
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 46 +++++++-
src/backend/utils/adt/pgstatfuncs.c | 100 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 13 ++-
src/include/pgstat.h | 5 +-
.../vacuum-extending-in-repetable-read.spec | 6 ++
src/test/regress/expected/rules.out | 17 +++
.../expected/vacuum_index_statistics.out | 16 +--
...ut => vacuum_tables_and_db_statistics.out} | 87 +++++++++++++--
src/test/regress/parallel_schedule | 2 +-
.../regress/sql/vacuum_index_statistics.sql | 6 +-
...ql => vacuum_tables_and_db_statistics.sql} | 69 +++++++++++-
16 files changed, 381 insertions(+), 35 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (82%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (81%)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index e206aeecc9c..3c3e23cd943 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -659,7 +659,7 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
@@ -4075,6 +4075,9 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4090,6 +4093,9 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4105,16 +4111,25 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index eb63be1833a..e5370211d74 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1501,4 +1501,29 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.errors AS errors
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 2b7058e5c01..614daf60372 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-bool pgstat_track_vacuum_statistics = true;
+bool pgstat_track_vacuum_statistics = false;
/* ----------
* state shared with pgstat_*.c
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index b31f20d41bc..65207d30378 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -485,6 +485,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index c6194584b35..bf7ab345be0 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -205,6 +205,38 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.type = m_type;
+ pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.errors++;
+ dbentry->vacuum_ext.type = m_type;
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
@@ -216,6 +248,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -274,6 +307,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
}
/*
@@ -1030,6 +1073,8 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->blk_write_time += src->blk_write_time;
dst->delay_time += src->delay_time;
dst->total_time += src->total_time;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->errors += src->errors;
if (!accumulate_reltype_specific_info)
return;
@@ -1057,7 +1102,6 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->table.index_vacuum_count += src->table.index_vacuum_count;
dst->table.missed_dead_pages += src->table.missed_dead_pages;
dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
}
else if (dst->type == PGSTAT_EXTVAC_INDEX)
{
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index defe1990e11..1c39ada2c3e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2385,7 +2385,7 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
@@ -2515,6 +2515,104 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index ac6427cf1e9..6f16db4bee9 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1574,7 +1574,7 @@ struct config_bool ConfigureNamesBool[] =
NULL
},
&pgstat_track_vacuum_statistics,
- true,
+ false,
NULL, NULL, NULL
},
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2fbd001178e..9781d4f59da 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12595,12 +12595,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,wraparound_failsafe,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index aec25ad2262..50876f5446f 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -154,6 +154,9 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
+ int32 errors;
+ int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+
ExtVacReportType type; /* heap, index, etc. */
/* ----------
@@ -183,7 +186,6 @@ typedef struct ExtVacReport
int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} table;
struct
{
@@ -762,6 +764,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5893d89573d..cfec3159580 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -18,6 +18,9 @@ teardown
}
session s1
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s1_begin_repeatable_read {
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
select count(ival) from test_vacuum_stat_isolation where id>900;
@@ -25,6 +28,9 @@ step s1_begin_repeatable_read {
step s1_commit { COMMIT; }
session s2
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index fdd5341bdfc..14f19e5bcbe 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2293,6 +2293,23 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schemaname,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index e00a0fc683c..9e5d33342c9 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -16,8 +16,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -33,12 +37,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -181,3 +180,4 @@ WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
(1 row)
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 82%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index b5ea9c9ab1e..0300e7b6276 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,7 +6,6 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
--------------
@@ -16,8 +15,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -37,12 +40,12 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -225,3 +228,69 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
(1 row)
DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4eb03353104..798692cf21a 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -145,4 +145,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index ae146e1d23f..9b7e645187d 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -14,8 +14,7 @@ SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -33,7 +32,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -149,3 +148,4 @@ FROM pg_stat_vacuum_indexes vt, pg_class c
WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 81%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 5bc34bec64b..ca7dbde9387 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,15 +7,13 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -36,7 +34,13 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -180,4 +184,59 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+DROP TABLE vestat CASCADE;
+
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
\ No newline at end of file
--
2.34.1
[text/x-patch] v24-0004-Vacuum-statistics-have-been-separated-from-regular.patch (93.8K, 5-v24-0004-Vacuum-statistics-have-been-separated-from-regular.patch)
download | inline diff:
From c7acebdd3a7b21b615c087aef9c6acd30e49c4a4 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 1 Sep 2025 21:44:59 +0300
Subject: [PATCH 4/5] Vacuum statistics have been separated from regular
relation and database statistics to reduce memory usage. Dedicated
PGSTAT_KIND_VACUUM_RELATION and PGSTAT_KIND_VACUUM_DB entries were added to
the stats collector to efficiently allocate memory for vacuum-specific
metrics, which require significantly more space per relation.
---
src/backend/access/heap/vacuumlazy.c | 196 ++++++------
src/backend/catalog/heap.c | 1 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/system_views.sql | 178 +++++------
src/backend/commands/dbcommands.c | 1 +
src/backend/commands/vacuumparallel.c | 14 +-
src/backend/utils/activity/Makefile | 1 +
src/backend/utils/activity/pgstat.c | 28 ++
src/backend/utils/activity/pgstat_database.c | 10 +-
src/backend/utils/activity/pgstat_relation.c | 111 +------
src/backend/utils/activity/pgstat_vacuum.c | 215 +++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 285 +++++-------------
src/include/commands/vacuum.h | 2 +-
src/include/pgstat.h | 200 ++++++------
src/include/utils/pgstat_internal.h | 15 +
src/include/utils/pgstat_kind.h | 4 +-
src/test/regress/expected/rules.out | 146 ++++-----
.../expected/vacuum_index_statistics.out | 82 ++---
.../regress/sql/vacuum_index_statistics.sql | 48 +--
19 files changed, 805 insertions(+), 733 deletions(-)
create mode 100644 src/backend/utils/activity/pgstat_vacuum.c
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3c3e23cd943..606a03827c7 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -412,7 +412,7 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
- ExtVacReport extVacReportIdx;
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -524,7 +524,7 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters *counters,
- ExtVacReport *report)
+ PgStat_CommonCounts *report)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -585,6 +585,8 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
if(!pgstat_track_vacuum_statistics)
return;
+ memset(&counters->common, 0, sizeof(LVExtStatCounters));
+
/* Set initial values for common heap and index statistics*/
extvac_stats_start(rel, &counters->common);
counters->pages_deleted = counters->tuples_removed = 0;
@@ -602,11 +604,13 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
void
extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report)
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report)
{
- memset(report, 0, sizeof(ExtVacReport));
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ extvac_stats_end(rel, &counters->common, &report->common);
- extvac_stats_end(rel, &counters->common, report);
report->type = PGSTAT_EXTVAC_INDEX;
if (stats != NULL)
@@ -617,7 +621,7 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
*/
/* Fill index-specific extended stats fields */
- report->tuples_deleted =
+ report->common.tuples_deleted =
stats->tuples_removed - counters->tuples_removed;
report->index.pages_deleted =
stats->pages_deleted - counters->pages_deleted;
@@ -640,7 +644,7 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
* procudure.
*/
static void
-accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacStats)
{
if (!pgstat_track_vacuum_statistics)
return;
@@ -652,49 +656,49 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
- extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
- extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
- extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
- extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
- extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
- extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
- extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
- extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
- extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
- extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
}
static void
-accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacIdxStats)
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacIdxStats)
{
if (!pgstat_track_vacuum_statistics)
return;
/* Fill heap-specific extended stats fields */
- vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
- vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
- vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
- vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
- vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
- vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
- vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
- vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
- vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
- vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
-
- vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
}
@@ -855,12 +859,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ErrorContextCallback errcallback;
char **indnames = NULL;
LVExtStatCounters extVacCounters;
- ExtVacReport extVacReport;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts extVacReport;
/* Initialize vacuum statistics */
- memset(&extVacReport, 0, sizeof(ExtVacReport));
- extVacReport = allzero;
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -906,7 +908,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
- memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+ memset(&vacrel->extVacReportIdx, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
@@ -1158,7 +1161,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
&frozenxid_updated, &minmulti_updated, false);
/* Make generic extended vacuum stats report */
- extvac_stats_end(rel, &extVacCounters, &extVacReport);
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
/*
* Report results to the cumulative stats system, too.
@@ -1170,33 +1173,20 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- if(pgstat_track_vacuum_statistics)
- {
- /* Make generic extended vacuum stats report and
- * fill heap-specific extended stats fields.
- */
- extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
- accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime,
- &extVacReport);
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
- }
- else
- {
- pgstat_report_vacuum(RelationGetRelid(rel),
+ pgstat_report_vacuum_extstats(vacrel->reloid, rel->rd_rel->relisshared, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime,
- NULL);
- }
+ starttime);
pgstat_progress_end_command();
@@ -2890,9 +2880,9 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -2900,7 +2890,7 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
- extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
/*
@@ -3313,9 +3303,9 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -3324,7 +3314,7 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
vacrel->num_index_scans,
estimated_count);
- extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
@@ -3352,7 +3342,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3383,18 +3376,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- if (!ParallelVacuumIsActive(vacrel))
- accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ pgstat_report_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -3421,7 +3409,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3451,17 +3442,13 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_cleanup_one_index(&ivinfo, istat);
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- if (!ParallelVacuumIsActive(vacrel))
- accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
-
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -4061,6 +4048,27 @@ update_relstats_all_indexes(LVRelState *vacrel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+static void
+pgstat_report_vacuum_error()
+{
+ PgStat_VacuumDBCounts *vacuum_dbentry;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ vacuum_dbentry = pgstat_fetch_stat_vacuum_dbentry(MyDatabaseId);
+
+ if(vacuum_dbentry == NULL)
+ return;
+ vacuum_dbentry->errors++;
+}
+
/*
* Error context callback for errors occurring during vacuum. The error
* context messages for index phases should match the messages set in parallel
@@ -4076,7 +4084,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -4094,7 +4102,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -4112,7 +4120,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+ pgstat_report_vacuum_error();
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
@@ -4120,7 +4128,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+ pgstat_report_vacuum_error();
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
@@ -4128,7 +4136,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_TRUNCATE:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index fd6537567ea..f82f11490ff 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1883,6 +1883,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c4029a4f3d3..3bccc1097fb 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2327,6 +2327,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index e5370211d74..b3f60ea546d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1430,100 +1430,104 @@ GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
--
CREATE VIEW pg_stat_vacuum_tables AS
-SELECT
- ns.nspname AS schemaname,
- rel.relname AS relname,
- stats.relid as relid,
-
- stats.total_blks_read AS total_blks_read,
- stats.total_blks_hit AS total_blks_hit,
- stats.total_blks_dirtied AS total_blks_dirtied,
- stats.total_blks_written AS total_blks_written,
-
- stats.rel_blks_read AS rel_blks_read,
- stats.rel_blks_hit AS rel_blks_hit,
-
- stats.pages_scanned AS pages_scanned,
- stats.pages_removed AS pages_removed,
- stats.vm_new_frozen_pages AS vm_new_frozen_pages,
- stats.vm_new_visible_pages AS vm_new_visible_pages,
- stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
- stats.missed_dead_pages AS missed_dead_pages,
- stats.tuples_deleted AS tuples_deleted,
- stats.tuples_frozen AS tuples_frozen,
- stats.recently_dead_tuples AS recently_dead_tuples,
- stats.missed_dead_tuples AS missed_dead_tuples,
-
- stats.wraparound_failsafe AS wraparound_failsafe,
- stats.index_vacuum_count AS index_vacuum_count,
- stats.wal_records AS wal_records,
- stats.wal_fpi AS wal_fpi,
- stats.wal_bytes AS wal_bytes,
-
- stats.blk_read_time AS blk_read_time,
- stats.blk_write_time AS blk_write_time,
-
- stats.delay_time AS delay_time,
- stats.total_time AS total_time
-
-FROM pg_class rel
- JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
- LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
-WHERE rel.relkind = 'r';
+ SELECT
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ S.relid as relid,
+
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
+
+ S.pages_scanned AS pages_scanned,
+ S.pages_removed AS pages_removed,
+ S.vm_new_frozen_pages AS vm_new_frozen_pages,
+ S.vm_new_visible_pages AS vm_new_visible_pages,
+ S.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ S.missed_dead_pages AS missed_dead_pages,
+ S.tuples_deleted AS tuples_deleted,
+ S.tuples_frozen AS tuples_frozen,
+ S.recently_dead_tuples AS recently_dead_tuples,
+ S.missed_dead_tuples AS missed_dead_tuples,
+
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.index_vacuum_count AS index_vacuum_count,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+
+ FROM pg_class C JOIN
+ pg_namespace N ON N.oid = C.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(C.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
CREATE VIEW pg_stat_vacuum_indexes AS
-SELECT
- rel.oid as relid,
- ns.nspname AS schemaname,
- rel.relname AS relname,
+ SELECT
+ C.oid AS relid,
+ I.oid AS indexrelid,
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ I.relname AS indexrelname,
- total_blks_read AS total_blks_read,
- total_blks_hit AS total_blks_hit,
- total_blks_dirtied AS total_blks_dirtied,
- total_blks_written AS total_blks_written,
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
- rel_blks_read AS rel_blks_read,
- rel_blks_hit AS rel_blks_hit,
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
- pages_deleted AS pages_deleted,
- tuples_deleted AS tuples_deleted,
+ S.pages_deleted AS pages_deleted,
+ S.tuples_deleted AS tuples_deleted,
- wal_records AS wal_records,
- wal_fpi AS wal_fpi,
- wal_bytes AS wal_bytes,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
- blk_read_time AS blk_read_time,
- blk_write_time AS blk_write_time,
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
- delay_time AS delay_time,
- total_time AS total_time
-FROM
- pg_class rel
- JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
- LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+ FROM
+ pg_class C JOIN
+ pg_index X ON C.oid = X.indrelid JOIN
+ pg_class I ON I.oid = X.indexrelid
+ LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace),
+ LATERAL pg_stat_get_vacuum_indexes(I.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
CREATE VIEW pg_stat_vacuum_database AS
-SELECT
- db.oid as dboid,
- db.datname AS dbname,
-
- stats.db_blks_read AS db_blks_read,
- stats.db_blks_hit AS db_blks_hit,
- stats.total_blks_dirtied AS total_blks_dirtied,
- stats.total_blks_written AS total_blks_written,
-
- stats.wal_records AS wal_records,
- stats.wal_fpi AS wal_fpi,
- stats.wal_bytes AS wal_bytes,
-
- stats.blk_read_time AS blk_read_time,
- stats.blk_write_time AS blk_write_time,
-
- stats.delay_time AS delay_time,
- stats.total_time AS total_time,
- stats.wraparound_failsafe AS wraparound_failsafe,
- stats.errors AS errors
-FROM
- pg_database db,
- LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
+ SELECT
+ D.oid as dboid,
+ D.datname AS dbname,
+
+ S.db_blks_read AS db_blks_read,
+ S.db_blks_hit AS db_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time,
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.errors AS errors
+ FROM
+ pg_database D,
+ LATERAL pg_stat_get_vacuum_database(D.oid) S;
\ No newline at end of file
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 2793fd83771..25ede1d9824 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1815,6 +1815,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 65de45a4447..3c37d1f07ce 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,7 +869,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -909,14 +909,10 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared,
+ &extVacReport);
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 9c2443e1ecd..183f7514d2d 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 614daf60372..d03fc10666e 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -479,6 +479,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 65207d30378..80e6c7c229a 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
@@ -485,7 +494,6 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
- memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bf7ab345be0..817372f9cec 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
@@ -205,50 +203,17 @@ pgstat_drop_relation(Relation rel)
}
}
-/* ---------
- * pgstat_report_vacuum_error() -
- *
- * Tell the collector about an (auto)vacuum interruption.
- * ---------
- */
-void
-pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
-{
- PgStat_EntryRef *entry_ref;
- PgStatShared_Relation *shtabentry;
- PgStat_StatTabEntry *tabentry;
- Oid dboid = MyDatabaseId;
- PgStat_StatDBEntry *dbentry; /* pending database entry */
-
- if (!pgstat_track_counts)
- return;
-
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
- dboid, tableoid, false);
-
- shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
- tabentry = &shtabentry->stats;
-
- tabentry->vacuum_ext.type = m_type;
- pgstat_unlock_entry(entry_ref);
-
- dbentry = pgstat_prep_database_pending(dboid);
- dbentry->vacuum_ext.errors++;
- dbentry->vacuum_ext.type = m_type;
-}
-
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params)
+ TimestampTz starttime)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
- PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -270,8 +235,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
- pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
-
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -307,16 +270,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
-
- if (dboid != InvalidOid)
- {
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dboid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
-
- pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
- pgstat_unlock_entry(entry_ref);
- }
}
/*
@@ -951,6 +904,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
@@ -1053,60 +1012,4 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_updated = trans->updated_pre_truncdrop;
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
-}
-
-static void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info)
-{
- if(!pgstat_track_vacuum_statistics)
- return;
-
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->total_time += src->total_time;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->errors += src->errors;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_TABLE)
- {
- dst->table.pages_scanned += src->table.pages_scanned;
- dst->table.pages_removed += src->table.pages_removed;
- dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
- dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
- dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->table.tuples_frozen += src->table.tuples_frozen;
- dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
- dst->table.index_vacuum_count += src->table.index_vacuum_count;
- dst->table.missed_dead_pages += src->table.missed_dead_pages;
- dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->tuples_deleted += src->tuples_deleted;
- }
- }
}
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat_vacuum.c b/src/backend/utils/activity/pgstat_vacuum.c
new file mode 100644
index 00000000000..e11f19e46b2
--- /dev/null
+++ b/src/backend/utils/activity/pgstat_vacuum.c
@@ -0,0 +1,215 @@
+#include "postgres.h"
+
+#include "pgstat.h"
+#include "utils/pgstat_internal.h"
+#include "utils/memutils.h"
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+bool pgstat_track_vacuum_statistics_for_relations = false;
+
+#define ACCUMULATE_FIELD(field) dst->field += src->field;
+
+#define ACCUMULATE_SUBFIELD(substruct, field) \
+ (dst->substruct.field += src->substruct.field)
+
+static void
+pgstat_accumulate_common(PgStat_CommonCounts *dst, const PgStat_CommonCounts *src)
+{
+ ACCUMULATE_FIELD(total_blks_read);
+ ACCUMULATE_FIELD(total_blks_hit);
+ ACCUMULATE_FIELD(total_blks_dirtied);
+ ACCUMULATE_FIELD(total_blks_written);
+
+ ACCUMULATE_FIELD(blks_fetched);
+ ACCUMULATE_FIELD(blks_hit);
+
+ ACCUMULATE_FIELD(wal_records);
+ ACCUMULATE_FIELD(wal_fpi);
+ ACCUMULATE_FIELD(wal_bytes);
+
+ ACCUMULATE_FIELD(blk_read_time);
+ ACCUMULATE_FIELD(blk_write_time);
+ ACCUMULATE_FIELD(delay_time);
+ ACCUMULATE_FIELD(total_time);
+
+ ACCUMULATE_FIELD(tuples_deleted);
+ ACCUMULATE_FIELD(wraparound_failsafe_count);
+}
+
+static void
+pgstat_accumulate_extvac_stats_relations(PgStat_VacuumRelationCounts *dst, PgStat_VacuumRelationCounts *src)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB && src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ ACCUMULATE_SUBFIELD(common, blks_fetched);
+ ACCUMULATE_SUBFIELD(common, blks_hit);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(table, pages_scanned);
+ ACCUMULATE_SUBFIELD(table, pages_removed);
+ ACCUMULATE_SUBFIELD(table, vm_new_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, tuples_frozen);
+ ACCUMULATE_SUBFIELD(table, recently_dead_tuples);
+ ACCUMULATE_SUBFIELD(table, index_vacuum_count);
+ ACCUMULATE_SUBFIELD(table, missed_dead_pages);
+ ACCUMULATE_SUBFIELD(table, missed_dead_tuples);
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(index, pages_deleted);
+ }
+}
+
+static void
+pgstat_accumulate_extvac_stats_db(PgStat_VacuumDBCounts *dst, PgStat_VacuumDBCounts *src)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+ dst->errors += src->errors;
+}
+
+/*
+ * Report that the table was just vacuumed and flush statistics.
+ */
+void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_VacuumRelation *shtabentry;
+ PgStatShared_VacuumDB *shdbentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_RELATION,
+ dboid, tableoid, false);
+ shtabentry = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+ pgstat_accumulate_extvac_stats_relations(&shtabentry->stats, params);
+
+ pgstat_unlock_entry(entry_ref);
+
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_DB,
+ dboid, InvalidOid, false);
+
+ shdbentry = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ pgstat_accumulate_common(&shdbentry->stats.common, ¶ms->common);
+
+ pgstat_unlock_entry(entry_ref);
+}
+
+/*
+ * Flush out pending stats for the entry
+ *
+ * If nowait is true, this function returns false if lock could not
+ * immediately acquired, otherwise true is returned.
+ */
+bool
+pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumRelation *shtabstats;
+ PgStat_RelationVacuumPending *pendingent; /* table entry of shared stats */
+
+ pendingent = (PgStat_RelationVacuumPending *) entry_ref->pending;
+ shtabstats = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts.
+ */
+ if (pg_memory_is_all_zeros(&pendingent,
+ sizeof(struct PgStat_RelationVacuumPending)))
+ return true;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ {
+ return false;
+ }
+
+ pgstat_accumulate_extvac_stats_relations(&(shtabstats->stats), &(pendingent->counts));
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the vacuum collected statistics for one relation or NULL.
+ */
+PgStat_VacuumRelationCounts *
+pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid)
+{
+ return (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_RELATION, dbid, relid);
+}
+
+PgStat_VacuumDBCounts *
+pgstat_fetch_stat_vacuum_dbentry(Oid dbid)
+{
+ return (PgStat_VacuumDBCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_DB, dbid, InvalidOid);
+}
+
+bool
+pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumDB *sharedent;
+ PgStat_VacuumDBCounts *pendingent;
+
+ pendingent = (PgStat_VacuumDBCounts *) entry_ref->pending;
+ sharedent = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ /* The entry was successfully flushed, add the same to database stats */
+ pgstat_accumulate_extvac_stats_db(&(sharedent->stats), pendingent);
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Find or create a local PgStat_VacuumDBCounts entry for dboid.
+ */
+PgStat_VacuumDBCounts *
+pgstat_prep_vacuum_database_pending(Oid dboid)
+{
+ PgStat_EntryRef *entry_ref;
+
+ /*
+ * This should not report stats on database objects before having
+ * connected to a database.
+ */
+ Assert(!OidIsValid(dboid) || OidIsValid(MyDatabaseId));
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_VACUUM_DB, dboid, InvalidOid,
+ NULL);
+
+ if(entry_ref == NULL)
+ return NULL;
+
+ return entry_ref->pending;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1c39ada2c3e..b2df237c337 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2267,7 +2267,6 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
-
/*
* Get the vacuum statistics for the heap tables.
*/
@@ -2277,102 +2276,42 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
-
- Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- tabentry = pgstat_fetch_stat_tabentry(relid);
-
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = pending;
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
@@ -2380,28 +2319,28 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
@@ -2418,100 +2357,60 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
-
- Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
-
- tabentry = pgstat_fetch_stat_tabentry(relid);
-
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = pending;
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
@@ -2525,90 +2424,52 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
Oid dbid = PG_GETARG_OID(0);
- PgStat_StatDBEntry *dbentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumDBCounts *extvacuum;
+ PgStat_VacuumDBCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
-
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
+ PgStat_VacuumDBCounts allzero;
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
- INT4OID, -1, 0);
+ pending = pgstat_fetch_stat_vacuum_dbentry(dbid);
- Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
-
- dbentry = pgstat_fetch_stat_dbentry(dbid);
-
- if (dbentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(dbentry->vacuum_ext);
- }
-
- i = 0;
+ extvacuum = pending;
values[i++] = ObjectIdGetDatum(dbid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
values[i++] = Int32GetDatum(extvacuum->errors);
Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index dcc542750b8..bc9df1433c2 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -432,5 +432,5 @@ extern double anl_get_next_S(double t, int n, double *stateptr);
extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx *counters);
extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report);
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 50876f5446f..3b9eb87221a 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -116,46 +116,100 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_TABLE = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
+ * PgStat_TableCounts The actual per-table counts kept by a backend
*
- * ExtVacReport
+ * This struct should contain only actual event counters, because we make use
+ * of pg_memory_is_all_zeros() to detect whether there are any stats updates
+ * to apply.
*
- * Additional statistics of vacuum processing over a relation.
- * pages_removed is the amount by which the physically shrank,
- * if any (ie the change in its total size on disk)
- * pages_deleted refer to free space within the index file
+ * It is a component of PgStat_TableStatus (within-backend state).
+ *
+ * Note: for a table, tuples_returned is the number of tuples successfully
+ * fetched by heap_getnext, while tuples_fetched is the number of tuples
+ * successfully fetched by heap_fetch under the control of bitmap indexscans.
+ * For an index, tuples_returned is the number of index entries returned by
+ * the index AM, while tuples_fetched is the number of tuples successfully
+ * fetched by heap_fetch under the control of simple indexscans for this index.
+ *
+ * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
+ * actions, regardless of whether the transaction committed. delta_live_tuples,
+ * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
+ * Note that delta_live_tuples and delta_dead_tuples can be negative!
* ----------
*/
-typedef struct ExtVacReport
+typedef struct PgStat_TableCounts
{
- /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
- int64 total_blks_read;
- int64 total_blks_hit;
- int64 total_blks_dirtied;
- int64 total_blks_written;
+ PgStat_Counter numscans;
- /* blocks missed and hit for just the heap during a vacuum of specific relation */
- int64 blks_fetched;
- int64 blks_hit;
+ PgStat_Counter tuples_returned;
+ PgStat_Counter tuples_fetched;
- /* Vacuum WAL usage stats */
- int64 wal_records; /* wal usage: number of WAL records */
- int64 wal_fpi; /* wal usage: number of WAL full page images produced */
- uint64 wal_bytes; /* wal usage: size of WAL records produced */
+ PgStat_Counter tuples_inserted;
+ PgStat_Counter tuples_updated;
+ PgStat_Counter tuples_deleted;
+ PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_newpage_updated;
+ bool truncdropped;
+
+ PgStat_Counter delta_live_tuples;
+ PgStat_Counter delta_dead_tuples;
+ PgStat_Counter changed_tuples;
+
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
- /* Time stats. */
- double blk_read_time; /* time spent reading pages, in msec */
- double blk_write_time; /* time spent writing pages, in msec */
- double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
- double total_time; /* total time of a vacuum operation, in msec */
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+} PgStat_TableCounts;
- int64 tuples_deleted; /* tuples deleted by vacuum */
+typedef struct PgStat_CommonCounts
+{
+ /* blocks */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* heap blocks */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* WAL */
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+
+ /* Time */
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+
+ /* tuples */
+ int64 tuples_deleted;
+
+ /* failsafe */
+ int32 wraparound_failsafe_count;
+} PgStat_CommonCounts;
- int32 errors;
- int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
ExtVacReportType type; /* heap, index, etc. */
@@ -174,16 +228,16 @@ typedef struct ExtVacReport
{
struct
{
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 pages_scanned; /* heap pages examined (not skipped by VM) */
int64 pages_removed; /* heap pages removed by vacuum "truncation" */
int64 pages_frozen; /* pages marked in VM as frozen */
int64 pages_all_visible; /* pages marked in VM as all-visible */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
} table;
@@ -192,61 +246,21 @@ typedef struct ExtVacReport
int64 pages_deleted; /* number of pages deleted by vacuum */
} index;
} /* per_type_stats */;
-} ExtVacReport;
+} PgStat_VacuumRelationCounts;
-/* ----------
- * PgStat_TableCounts The actual per-table counts kept by a backend
- *
- * This struct should contain only actual event counters, because we make use
- * of pg_memory_is_all_zeros() to detect whether there are any stats updates
- * to apply.
- *
- * It is a component of PgStat_TableStatus (within-backend state).
- *
- * Note: for a table, tuples_returned is the number of tuples successfully
- * fetched by heap_getnext, while tuples_fetched is the number of tuples
- * successfully fetched by heap_fetch under the control of bitmap indexscans.
- * For an index, tuples_returned is the number of index entries returned by
- * the index AM, while tuples_fetched is the number of tuples successfully
- * fetched by heap_fetch under the control of simple indexscans for this index.
- *
- * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
- * actions, regardless of whether the transaction committed. delta_live_tuples,
- * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
- * Note that delta_live_tuples and delta_dead_tuples can be negative!
- * ----------
- */
-typedef struct PgStat_TableCounts
+typedef struct PgStat_VacuumRelationStatus
{
- PgStat_Counter numscans;
-
- PgStat_Counter tuples_returned;
- PgStat_Counter tuples_fetched;
-
- PgStat_Counter tuples_inserted;
- PgStat_Counter tuples_updated;
- PgStat_Counter tuples_deleted;
- PgStat_Counter tuples_hot_updated;
- PgStat_Counter tuples_newpage_updated;
- bool truncdropped;
-
- PgStat_Counter delta_live_tuples;
- PgStat_Counter delta_dead_tuples;
- PgStat_Counter changed_tuples;
-
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-
- PgStat_Counter rev_all_visible_pages;
- PgStat_Counter rev_all_frozen_pages;
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
- /*
- * Additional cumulative stat on vacuum operations.
- * Use an expensive structure as an abstraction for different types of
- * relations.
- */
- ExtVacReport vacuum_ext;
-} PgStat_TableCounts;
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ PgStat_CommonCounts common;
+ int32 errors;
+} PgStat_VacuumDBCounts;
/* ----------
* PgStat_TableStatus Per-table status within a backend
@@ -272,6 +286,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -468,8 +488,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
-
- ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -551,8 +569,6 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter rev_all_visible_pages;
PgStat_Counter rev_all_frozen_pages;
-
- ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -760,11 +776,10 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params);
+ TimestampTz starttime);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
-extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -895,6 +910,15 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts *pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts *pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid);
+PgStat_VacuumDBCounts *pgstat_fetch_stat_vacuum_dbentry(Oid dbid);
/*
* Functions in pgstat_wal.c
*/
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 6cf00008f63..5c5ab8e2ee4 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -432,6 +432,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -600,6 +612,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index eb5f0b3ae6d..52e884fbf8b 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 14f19e5bcbe..ef2d6545953 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2293,77 +2293,81 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
-pg_stat_vacuum_database| SELECT db.oid AS dboid,
- db.datname AS dbname,
- stats.db_blks_read,
- stats.db_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time,
- stats.wraparound_failsafe,
- stats.errors
- FROM pg_database db,
- LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
-pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
- ns.nspname AS schemaname,
- rel.relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_deleted,
- stats.tuples_deleted,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time
- FROM (pg_class rel
- JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
- LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
- WHERE (rel.relkind = 'i'::"char");
-pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
- rel.relname,
- stats.relid,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_scanned,
- stats.pages_removed,
- stats.vm_new_frozen_pages,
- stats.vm_new_visible_pages,
- stats.vm_new_visible_frozen_pages,
- stats.missed_dead_pages,
- stats.tuples_deleted,
- stats.tuples_frozen,
- stats.recently_dead_tuples,
- stats.missed_dead_tuples,
- stats.wraparound_failsafe,
- stats.index_vacuum_count,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time
- FROM (pg_class rel
- JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
- LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
- WHERE (rel.relkind = 'r'::"char");
+pg_stat_vacuum_database| SELECT d.oid AS dboid,
+ d.datname AS dbname,
+ s.db_blks_read,
+ s.db_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time,
+ s.wraparound_failsafe,
+ s.errors
+ FROM pg_database d,
+ LATERAL pg_stat_get_vacuum_database(d.oid) s(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
+pg_stat_vacuum_indexes| SELECT c.oid AS relid,
+ i.oid AS indexrelid,
+ n.nspname AS schemaname,
+ c.relname,
+ i.relname AS indexrelname,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_deleted,
+ s.tuples_deleted,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (((pg_class c
+ JOIN pg_index x ON ((c.oid = x.indrelid)))
+ JOIN pg_class i ON ((i.oid = x.indexrelid)))
+ LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(i.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
+pg_stat_vacuum_tables| SELECT n.nspname AS schemaname,
+ c.relname,
+ s.relid,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_scanned,
+ s.pages_removed,
+ s.vm_new_frozen_pages,
+ s.vm_new_visible_pages,
+ s.vm_new_visible_frozen_pages,
+ s.missed_dead_pages,
+ s.tuples_deleted,
+ s.tuples_frozen,
+ s.recently_dead_tuples,
+ s.missed_dead_tuples,
+ s.wraparound_failsafe,
+ s.index_vacuum_count,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (pg_class c
+ JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(c.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index 9e5d33342c9..4654a536ad6 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -30,9 +30,9 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- Must be empty.
SELECT *
FROM pg_stat_vacuum_indexes vt
-WHERE vt.relname = 'vestat';
- relid | schemaname | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
--------+------------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+WHERE vt.indexrelname = 'vestat';
+ relid | indexrelid | schemaname | relname | indexrelname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+------------+------------+---------+--------------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
(0 rows)
RESET track_vacuum_statistics;
@@ -55,12 +55,12 @@ ANALYZE vestat;
SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
DELETE FROM vestat WHERE x % 2 = 0;
-- Before the first vacuum execution extended stats view is empty.
-SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+SELECT vt.indexrelname,relpages,pages_deleted,tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | 30 | 0 | 0
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
(1 row)
SELECT relpages AS irp
@@ -72,22 +72,22 @@ CHECKPOINT;
-- The table and index extended vacuum statistics should show us that
-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
-- because of not full table have cleaned up
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | t | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
(1 row)
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- Look into WAL records deltas.
SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey';
diwr | diwb
------+------
t | t
@@ -98,20 +98,20 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
-- it is necessary to check the wal statistics
CHECKPOINT;
-- pages_removed must be increased
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | t | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
(1 row)
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL advance should be detected.
SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
diwr | diwb
@@ -120,7 +120,7 @@ SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
(1 row)
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
DELETE FROM vestat WHERE x % 2 = 0;
-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
@@ -130,7 +130,7 @@ VACUUM FULL vestat;
CHECKPOINT;
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL and other statistics advance should not be detected.
SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
diwr | ifpi | diwb
@@ -138,19 +138,19 @@ SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
t | t | t
(1 row)
-SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | t | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
(1 row)
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
DELETE FROM vestat;
TRUNCATE vestat;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
@@ -158,7 +158,7 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
CHECKPOINT;
-- Store WAL advances into variables after removing all tuples from the table
SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
--There are nothing changed
SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
diwr | ifpi | diwb
@@ -171,12 +171,12 @@ SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
-- in vacuum extended statistics.
-- The pages_frozen, pages_scanned values shouldn't be changed
--
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | f | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
(1 row)
DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index 9b7e645187d..57e5420b9b6 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -27,7 +27,7 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- Must be empty.
SELECT *
FROM pg_stat_vacuum_indexes vt
-WHERE vt.relname = 'vestat';
+WHERE vt.indexrelname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
@@ -49,9 +49,9 @@ SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
DELETE FROM vestat WHERE x % 2 = 0;
-- Before the first vacuum execution extended stats view is empty.
-SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+SELECT vt.indexrelname,relpages,pages_deleted,tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
SELECT relpages AS irp
FROM pg_class c
WHERE relname = 'vestat_pkey' \gset
@@ -63,19 +63,19 @@ CHECKPOINT;
-- The table and index extended vacuum statistics should show us that
-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
-- because of not full table have cleaned up
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- Look into WAL records deltas.
SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey';
DELETE FROM vestat;;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
@@ -83,22 +83,22 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
CHECKPOINT;
-- pages_removed must be increased
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL advance should be detected.
SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
DELETE FROM vestat WHERE x % 2 = 0;
@@ -110,20 +110,20 @@ CHECKPOINT;
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL and other statistics advance should not be detected.
SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
-SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
DELETE FROM vestat;
TRUNCATE vestat;
@@ -133,7 +133,7 @@ CHECKPOINT;
-- Store WAL advances into variables after removing all tuples from the table
SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
--There are nothing changed
SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
@@ -143,9 +143,9 @@ SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
-- in vacuum extended statistics.
-- The pages_frozen, pages_scanned values shouldn't be changed
--
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
DROP TABLE vestat;
RESET track_vacuum_statistics;
--
2.34.1
[text/x-patch] v24-0005-Add-documentation-about-the-system-views-that-are-us.patch (24.5K, 6-v24-0005-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 1a7be3fe1e1232c800e1c20210e27c2279155d3b Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 5/5] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 755 +++++++++++++++++++++++++++++++++
1 file changed, 755 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 4187191ea74..183fb0bfce3 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5545,4 +5545,759 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.34.1
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-09-04 15:49 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2025-09-04 15:49 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Amit Kapila <[email protected]>; pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
Hi, all!
On 02.06.2025 19:50, Alena Rybakina wrote:
>
> On 02.06.2025 19:25, Alexander Korotkov wrote:
>> On Tue, May 13, 2025 at 12:49 PM Alena Rybakina
>> <[email protected]> wrote:
>>> On 12.05.2025 08:30, Amit Kapila wrote:
>>>> On Fri, May 9, 2025 at 5:34 PM Alena Rybakina
>>>> <[email protected]> wrote:
>>>>> I did a rebase and finished the part with storing statistics
>>>>> separately from the relation statistics - now it is possible to
>>>>> disable the collection of statistics for relationsh using gucs and
>>>>> this allows us to solve the problem with the memory consumed.
>>>>>
>>>> I think this patch is trying to collect data similar to what we do for
>>>> pg_stat_statements for SQL statements. So, can't we follow a similar
>>>> idea such that these additional statistics will be collected once some
>>>> external module like pg_stat_statements is enabled? That module should
>>>> be responsible for accumulating and resetting the data, so we won't
>>>> have this memory consumption issue.
>>> The idea is good, it will require one hook for the pgstat_report_vacuum
>>> function, the extvac_stats_start and extvac_stats_end functions can be
>>> run if the extension is loaded, so as not to add more hooks.
>> +1
>> Nice idea of a hook. Given the volume of the patch, it might be a
>> good idea to keep this as an extension.
> Okay, I'll realize it and apply the patch)
>>
>>> But I see a problem here with tracking deleted objects for which
>>> statistics are no longer needed. There are two solutions to this and I
>>> don't like both of them, to be honest.
>>> The first way is to add a background process that will go through the
>>> table with saved statistics and check whether the relation or the
>>> database are relevant now or not and if not, then
>>> delete the vacuum statistics information for it. This may be
>>> resource-intensive. The second way is to add hooks for deleting the
>>> database and relationships (functions dropdb, index_drop,
>>> heap_drop_with_catalog).
>> Can we workaround this with object_access_hook?
>
> I think this could fix the problem. For the OAT-DROP access type, we
> can call a function to reset the vacuum statistics for relations that
> are about to be dropped.
>
> At the moment, I don’t see any limitations to using this approach.
>
I’ve prepared the first working version of the extension.
I haven’t yet implemented writing the statistics to a file and reloading
them into a hash table and shared memory at instance startup, and I also
haven’t implemented a proper output for database-level statistics yet.
I structured the extension as follows: statistics are stored in a hash
table keyed by a composite key - database OID, relation OID, and object
type (index, table, or database). When VACUUM or a worker processes a
table or index, an exclusive lock is taken to update the corresponding
record; a shared lock is taken when reading the statistics. For
database-level output, I plan to compute the totals by summing table and
index statistics on demand.
To optimize that, I plan to keep entries in the hash table ordered by
database OID. When accessing the first element by the partial key
(database OID), I’ll scan forward and aggregate until the partitial
database key changes.
Right now this requires adding the extension to
`shared_preload_libraries`. I haven’t found a way to avoid that because
of shared-memory setup, and I’m not sure it’s even possible.
I’m also unsure whether it’s better to store the statistics in the
cumulative statistics system (as done here) or entirely inside the
extension. Note that the code added to the core to support the extension
executes regardless of whether the extension is enabled.
Attachments:
[text/x-patch] 0001-Core-patch-for-vacuum-statistics.patch (37.3K, 2-0001-Core-patch-for-vacuum-statistics.patch)
download | inline diff:
From 098e381ca88e2600cb8e8528e3f54f588d4c43a8 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 4 Sep 2025 18:16:52 +0300
Subject: [PATCH] Core patch for vacuum statistics.
---
src/backend/access/heap/vacuumlazy.c | 309 ++++++++++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 12 +
src/backend/utils/activity/pgstat_relation.c | 36 ++-
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/backend/utils/error/elog.c | 13 +
src/include/catalog/pg_proc.dat | 8 +
src/include/commands/vacuum.h | 26 ++
src/include/pgstat.h | 119 ++++++-
src/include/utils/elog.h | 1 +
src/test/regress/expected/rules.out | 12 +-
12 files changed, 545 insertions(+), 11 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 932701d8420..330f9f90a6b 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -289,6 +289,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -407,6 +409,10 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -474,6 +480,209 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ PgStat_VacuumRelationCounts *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->common.total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->common.total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->common.total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->common.total_blks_written += bufusage.shared_blks_written;
+
+ report->common.wal_records += walusage.wal_records;
+ report->common.wal_fpi += walusage.wal_fpi;
+ report->common.wal_bytes += walusage.wal_bytes;
+
+ report->common.blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->common.blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->common.blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->common.blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->common.delay_time += VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->common.total_time += secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->common.blks_fetched +=
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->common.blks_hit +=
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
+
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ /* Set initial values for common heap and index statistics*/
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacStats)
+{
+ /* Fill heap-specific extended stats fields */
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacStats->table.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+
+}
+
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacIdxStats)
+{
+
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
/*
@@ -632,6 +841,13 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
char **indnames = NULL;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts ExtVacReport;
+ PgStat_VacuumRelationCounts allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
+ ExtVacReport = allzero;
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -652,6 +868,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -668,6 +885,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -676,6 +894,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(PgStat_VacuumRelationCounts));
+
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
vac_open_indexes(vacrel->rel, RowExclusiveLock, &vacrel->nindexes,
@@ -776,6 +996,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->aggressive = vacuum_get_cutoffs(rel, params, &vacrel->cutoffs);
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ vacrel->wraparound_failsafe_count = 0;
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
@@ -924,6 +1145,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &ExtVacReport);
+
/*
* Report results to the cumulative stats system, too.
*
@@ -934,12 +1158,20 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &ExtVacReport);
+ accumulate_heap_vacuum_statistics(vacrel, &ExtVacReport);
+
pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ rel->rd_rel->relisshared,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &ExtVacReport);
+
pgstat_progress_end_command();
if (instrument)
@@ -2631,10 +2863,20 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ PgStat_VacuumRelationCounts PgStat_VacuumRelationCounts;
+
+ memset(&PgStat_VacuumRelationCounts, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
+ extvac_stats_end(vacrel->rel, &counters, &PgStat_VacuumRelationCounts);
+ accumulate_idxs_vacuum_statistics(vacrel, &PgStat_VacuumRelationCounts);
+
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
* that parallel VACUUM only gets the precheck and this postcheck.
@@ -2961,6 +3203,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[2] = {0, 0};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count ++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3043,10 +3286,20 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ PgStat_VacuumRelationCounts PgStat_VacuumRelationCounts;
+
+ memset(&PgStat_VacuumRelationCounts, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
estimated_count);
+
+ extvac_stats_end(vacrel->rel, &counters, &PgStat_VacuumRelationCounts);
+ accumulate_idxs_vacuum_statistics(vacrel, &PgStat_VacuumRelationCounts);
}
/* Reset the progress counters */
@@ -3072,6 +3325,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts PgStat_VacuumRelationCounts;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3090,6 +3348,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3098,6 +3357,16 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &PgStat_VacuumRelationCounts);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &PgStat_VacuumRelationCounts);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &PgStat_VacuumRelationCounts);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3122,6 +3391,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts PgStat_VacuumRelationCounts;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3141,12 +3415,22 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &PgStat_VacuumRelationCounts);
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &PgStat_VacuumRelationCounts);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &PgStat_VacuumRelationCounts);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3759,6 +4043,9 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, errinfo->rel->rd_rel->relisshared, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3774,6 +4061,9 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, errinfo->rel->rd_rel->relisshared, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3789,16 +4079,25 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, errinfo->rel->rd_rel->relisshared, PGSTAT_EXTVAC_INDEX);
+
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, errinfo->rel->rd_rel->relisshared, PGSTAT_EXTVAC_INDEX);
+
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, errinfo->rel->rd_rel->relisshared, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 953ad4a4843..a21e77cd551 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 733ef40ae7c..d8776ff1901 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -116,6 +116,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2533,6 +2536,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 0feea1d30ec..b5461ec661b 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,12 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1054,6 +1065,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 69df741cbf6..33a4009f746 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -203,13 +203,39 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, bool shared, ExtVacReportType m_type)
+{
+ PgStat_VacuumRelationCounts params;
+
+ if (!pgstat_track_counts)
+ return;
+
+ if (set_report_vacuum_hook)
+ {
+ memset(¶ms, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ params.common.interrupts_count++;
+
+ (*set_report_vacuum_hook) (tableoid, shared, ¶ms);
+ }
+}
+
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime)
+ TimestampTz starttime, PgStat_VacuumRelationCounts *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -235,6 +261,11 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ if (set_report_vacuum_hook)
+ {
+ (*set_report_vacuum_hook) (tableoid, shared, params);
+ }
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -881,6 +912,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..9482bf80721 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index b7b9692f8c8..f0ecf86e514 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1627,6 +1627,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 118d6da1ace..e0c7cf29b3a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12576,4 +12576,12 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 14eeccbd718..bc9df1433c2 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -295,6 +296,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -327,6 +348,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
@@ -407,4 +429,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f402b17295c..ed6b3dc1d6f 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -100,6 +100,98 @@ typedef struct PgStat_FunctionCallUsage
instr_time start;
} PgStat_FunctionCallUsage;
+
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ /* blocks */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* heap blocks */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* WAL */
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+
+ /* Time */
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+
+ /* failsafe */
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+} PgStat_CommonCounts;
+
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ int64 tuples_deleted;
+ } table;
+ struct
+ {
+ int64 tuples_deleted;
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
+} PgStat_VacuumRelationCounts;
+
+
/* ----------
* PgStat_BackendSubEntry Non-flushed subscription stats.
* ----------
@@ -153,6 +245,9 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -211,7 +306,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB7
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB8
typedef struct PgStat_ArchiverStats
{
@@ -453,6 +548,9 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autovacuum_time;
PgStat_Counter total_analyze_time;
PgStat_Counter total_autoanalyze_time;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_StatTabEntry;
/* ------
@@ -660,10 +758,11 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime);
+ TimestampTz starttime, PgStat_VacuumRelationCounts *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
+extern void pgstat_report_vacuum_error(Oid tableoid, bool shared, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -711,6 +810,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -838,4 +948,9 @@ extern PGDLLIMPORT PgStat_Counter pgStatTransactionIdleTime;
/* updated by the traffic cop and in errfinish() */
extern PGDLLIMPORT SessionEndType pgStatSessionEndCause;
+/* Hook for plugins to get control in set_rel_pathlist() */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared, PgStat_VacuumRelationCounts *params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
+
+
#endif /* PGSTAT_H */
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 675f4f5f469..356dadd6b0a 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrcode(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..4731ca2121e 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1833,7 +1833,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_vacuum_time(c.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2232,7 +2234,9 @@ pg_stat_sys_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2284,7 +2288,9 @@ pg_stat_user_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.34.1
[text/x-patch] 0001-Create-vacuum-extension-statistics.patch (75.1K, 3-0001-Create-vacuum-extension-statistics.patch)
download | inline diff:
From 9b4dd2e845cb8fdc9c1cee09da8665da2bccfd42 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 4 Sep 2025 18:24:46 +0300
Subject: [PATCH] Create vacuum extension statistics
---
Makefile | 21 +
.../vacuum-extending-in-repetable-read.out | 53 ++
expected/vacuum_index_statistics.out | 183 +++++
expected/vacuum_tables_and_db_statistics.out | 296 ++++++++
spec/vacuum-extending-in-repetable-read.spec | 173 +++++
sql/vacuum_index_statistics.sql | 138 ++++
sql/vacuum_tables_and_db_statistics.sql | 224 ++++++
vacuum_statistics--1.0.sql | 191 ++++++
vacuum_statistics.c | 636 ++++++++++++++++++
vacuum_statistics.control | 4 +
10 files changed, 1919 insertions(+)
create mode 100644 Makefile
create mode 100644 expected/vacuum-extending-in-repetable-read.out
create mode 100644 expected/vacuum_index_statistics.out
create mode 100644 expected/vacuum_tables_and_db_statistics.out
create mode 100644 spec/vacuum-extending-in-repetable-read.spec
create mode 100644 sql/vacuum_index_statistics.sql
create mode 100644 sql/vacuum_tables_and_db_statistics.sql
create mode 100644 vacuum_statistics--1.0.sql
create mode 100644 vacuum_statistics.c
create mode 100644 vacuum_statistics.control
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..7fd875e
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,21 @@
+EXTENSION = vacuum_statistics
+EXTVERSION = 1.0
+MODULE_big = vacuum_statistics
+PGFILEDESC = "Vacuum Statistics - extension for storage statistics of vacuum workload"
+OBJS = vacuum_statistics.o
+
+DATA = vacuum_statistics--1.0.sql
+
+REGRESS = vacuum_index_statistics vacuum_tables_and_db_statistics
+ISOLATION = vacuum-extending-in-repetable-read
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
\ No newline at end of file
diff --git a/expected/vacuum-extending-in-repetable-read.out b/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 0000000..6d96042
--- /dev/null
+++ b/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
+(1 row)
+
diff --git a/expected/vacuum_index_statistics.out b/expected/vacuum_index_statistics.out
new file mode 100644
index 0000000..4654a53
--- /dev/null
+++ b/expected/vacuum_index_statistics.out
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.indexrelname = 'vestat';
+ relid | indexrelid | schemaname | relname | indexrelname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+------------+------------+---------+--------------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+(0 rows)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SET track_vacuum_statistics TO 'on';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.indexrelname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.indexrelname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/expected/vacuum_tables_and_db_statistics.out b/expected/vacuum_tables_and_db_statistics.out
new file mode 100644
index 0000000..0300e7b
--- /dev/null
+++ b/expected/vacuum_tables_and_db_statistics.out
@@ -0,0 +1,296 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+ relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_scanned | pages_removed | vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | missed_dead_pages | tuples_deleted | tuples_frozen | recently_dead_tuples | missed_dead_tuples | index_vacuum_count | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+---------------+---------------------+----------------------+-----------------------------+-------------------+----------------+---------------+----------------------+--------------------+--------------------+-------------+---------+-----------+---------------+----------------+------------+------------
+ vestat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
+(1 row)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
+---------------------+----------------------+----------------------+-----------------------+-----------------------------
+ 0 | 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | f | f
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
diff --git a/spec/vacuum-extending-in-repetable-read.spec b/spec/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 0000000..13b6c91
--- /dev/null
+++ b/spec/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,173 @@
+# A number of tests dedicated to verification of the 'Extended Vacuum Statistics'
+# feature.
+# By default, statistics has a volatile nature. So, selection result can depend
+# on a bunch of things. Here some trivial tests are performed that should work
+# in the most cases.
+# Test for checking pages: frozen, scanned, removed, number of tuple_deleted in pgpro_stats_vacuum_tables.
+# Besides, this test check pages scanned, pages removed, tuples_deleted in pgpro_stats_vacuum_tables and
+# wal values statistic collected over vacuum operation as for tables as for indexes.
+
+setup
+{
+ CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off);
+
+ CREATE EXTENSION vacuum_statistics;
+
+ CREATE TABLE vacuum_wal_stats_table
+ (relid int, wal_records int, wal_fpi int, wal_bytes int);
+ insert into vacuum_wal_stats_table (relid)
+ select oid from pg_class c
+ WHERE relname = 'vestat';
+ UPDATE vacuum_wal_stats_table SET
+ wal_records = 0, wal_fpi = 0, wal_bytes = 0;
+
+ CREATE TABLE vacuum_wal_stats_index
+ (relid int, wal_records int, wal_fpi int, wal_bytes int);
+ insert into vacuum_wal_stats_index (relid)
+ select oid from pg_class c
+ WHERE relname = 'vestat_pkey';
+ UPDATE vacuum_wal_stats_index SET
+ wal_records = 0, wal_fpi = 0, wal_bytes = 0;
+
+ SET track_io_timing = on;
+ SHOW track_counts; -- must be on
+ SET track_functions TO 'all';
+
+}
+
+teardown
+{
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+ DROP TABLE vestat CASCADE;
+ DROP TABLE vacuum_wal_stats_index;
+ DROP TABLE vacuum_wal_stats_table;
+ DROP EXTENSION vacuum_statistics;
+}
+
+session s1
+step s1_set_agressive_vacuum { SET vacuum_freeze_min_age = 0; }
+step s1_insert { INSERT INTO vestat(x) SELECT id FROM generate_series(1,770) As id; }
+step s1_update { UPDATE vestat SET x = x+1; }
+step s1_delete_half_table { DELETE FROM vestat WHERE x % 2 = 0; }
+step s1_delete_full_table { DELETE FROM vestat; }
+step s1_vacuum { VACUUM vestat; }
+step s1_vacuum_full { VACUUM FULL vestat; }
+step s1_vacuum_parallel { VACUUM (PARALLEL 2, INDEX_CLEANUP ON) vestat; }
+step s1_analyze { ANALYZE vestat; }
+step s1_trancate { TRUNCATE vestat; }
+step s1_checkpoint { CHECKPOINT; }
+step s1_print_vacuum_stats_tables
+{
+ SELECT vt.relname,
+ pages_frozen,
+ tuples_deleted,
+ pages_scanned,
+ pages_removed
+ FROM pgpro_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid;
+}
+
+step s1_print_vacuum_stats_indexes
+{
+ SELECT vt.relname,
+ pages_deleted,
+ tuples_deleted
+ FROM pgpro_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid;
+}
+
+step s1_save_walls
+{
+ UPDATE vacuum_wal_stats_table SET
+ wal_records = dWR, wal_fpi = dFPI, wal_bytes = dWB
+ FROM (SELECT relid, wal_records AS dWR, wal_fpi AS dFPI, wal_bytes AS dWB
+ FROM pgpro_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+ UPDATE vacuum_wal_stats_index SET
+ wal_records = iWR, wal_fpi = iFPI, wal_bytes = iWB
+ FROM (SELECT relid, wal_records AS iWR, wal_fpi AS iFPI, wal_bytes AS iWB
+ FROM pgpro_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+}
+
+step s1_difference
+{
+ SELECT t1.wal_records - t0.wal_records > 0 AS dWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS dFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS dWB
+ FROM vacuum_wal_stats_table t0, pgpro_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+ SELECT t1.wal_records - t0.wal_records > 0 AS iWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS iFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS iWB
+ FROM vacuum_wal_stats_table t0, pgpro_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+}
+
+permutation
+ s1_insert
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+ s1_set_agressive_vacuum
+ s1_analyze
+ s1_delete_half_table
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+ s1_vacuum
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+ s1_delete_full_table
+ s1_vacuum_parallel
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+ s1_insert
+ s1_update
+ s1_vacuum_full
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+ s1_delete_full_table
+ s1_trancate
+ s1_vacuum
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+
+permutation
+ s1_insert
+ s1_set_agressive_vacuum
+ s1_analyze
+ s1_delete_half_table
+ s1_checkpoint
+ s1_difference
+ s1_save_walls
+ s1_vacuum
+ s1_checkpoint
+ s1_difference
+ s1_save_walls
+ s1_delete_full_table
+ s1_vacuum_parallel
+ s1_checkpoint
+ s1_difference
+ s1_save_walls
+ s1_insert
+ s1_update
+ s1_vacuum_full
+ s1_checkpoint
+ s1_difference
+ s1_save_walls
+ s1_delete_full_table
+ s1_trancate
+ s1_vacuum
+ s1_checkpoint
+ s1_difference
+ s1_save_walls
\ No newline at end of file
diff --git a/sql/vacuum_index_statistics.sql b/sql/vacuum_index_statistics.sql
new file mode 100644
index 0000000..996ea04
--- /dev/null
+++ b/sql/vacuum_index_statistics.sql
@@ -0,0 +1,138 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+CREATE EXTENSION vacuum_statistics;
+
+SHOW vacuum_statistics.enabled; -- must be on
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT *
+FROM pg_stats_vacuum_indexes vt
+WHERE vt.indexrelname = 'vestat';
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.indexrelname,relpages,pages_deleted,tuples_deleted
+FROM pg_stats_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stats_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stats_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stats_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stats_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.indexrelname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stats_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stats_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stats_vacuum_indexes vt, pg_class c
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+
+DROP TABLE vestat;
+
+DROP EXTENSION vacuum_statistics;
diff --git a/sql/vacuum_tables_and_db_statistics.sql b/sql/vacuum_tables_and_db_statistics.sql
new file mode 100644
index 0000000..1e81a82
--- /dev/null
+++ b/sql/vacuum_tables_and_db_statistics.sql
@@ -0,0 +1,224 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+SHOW track_counts; -- must be on
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+CREATE EXTENSION vacuum_statistics;
+
+SHOW vacuum_statistics.enabled; -- must be on
+
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+CREATE EXTENSION vacuum_statistics;
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stats_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stats_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stats_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stats_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stats_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stats_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stats_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stats_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stats_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stats_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stats_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stats_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stats_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stats_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stats_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stats_vacuum_tables, pg_stat_all_tables WHERE pg_stats_vacuum_tables.relname = 'vestat' and pg_stats_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stats_vacuum_tables, pg_stat_all_tables WHERE pg_stats_vacuum_tables.relname = 'vestat' and pg_stats_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stats_vacuum_tables, pg_stat_all_tables WHERE pg_stats_vacuum_tables.relname = 'vestat' and pg_stats_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stats_vacuum_tables, pg_stat_all_tables WHERE pg_stats_vacuum_tables.relname = 'vestat' and pg_stats_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stats_vacuum_tables, pg_stat_all_tables WHERE pg_stats_vacuum_tables.relname = 'vestat' and pg_stats_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stats_vacuum_tables, pg_stat_all_tables WHERE pg_stats_vacuum_tables.relname = 'vestat' and pg_stats_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+DROP TABLE vestat CASCADE;
+
+select count(*) from pg_stats_vacuum_tables where relname = 'vestat';
+
+-- -- Now check vacuum statistics for current database
+-- SELECT dbname,
+-- db_blks_hit > 0 AS db_blks_hit,
+-- total_blks_dirtied > 0 AS total_blks_dirtied,
+-- total_blks_written > 0 AS total_blks_written,
+-- wal_records > 0 AS wal_records,
+-- wal_fpi > 0 AS wal_fpi,
+-- wal_bytes > 0 AS wal_bytes,
+-- total_time > 0 AS total_time
+-- FROM
+-- pg_stat_vacuum_database
+-- WHERE dbname = current_database();
+
+-- -- ensure pending stats are flushed
+-- SELECT pg_stat_force_next_flush();
+
+-- CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+-- INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+-- ANALYZE vestat;
+-- UPDATE vestat SET x = 10001;
+-- VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- \c regression_statistic_vacuum_db1;
+-- CREATE EXTENSION vacuum_statistics;
+
+-- -- Now check vacuum statistics for postgres database from another database
+-- SELECT dbname,
+-- db_blks_hit > 0 AS db_blks_hit,
+-- total_blks_dirtied > 0 AS total_blks_dirtied,
+-- total_blks_written > 0 AS total_blks_written,
+-- wal_records > 0 AS wal_records,
+-- wal_fpi > 0 AS wal_fpi,
+-- wal_bytes > 0 AS wal_bytes,
+-- total_time > 0 AS total_time
+-- FROM
+-- pg_stat_vacuum_database
+-- WHERE dbname = 'regression_statistic_vacuum_db';
+
+-- \c regression_statistic_vacuum_db
+
+-- DROP TABLE vestat CASCADE;
+
+-- \c regression_statistic_vacuum_db1;
+-- SELECT count(*)
+-- FROM pg_database d
+-- CROSS JOIN pg_stat_get_vacuum_tables(0)
+-- WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
\ No newline at end of file
diff --git a/vacuum_statistics--1.0.sql b/vacuum_statistics--1.0.sql
new file mode 100644
index 0000000..bb78f3a
--- /dev/null
+++ b/vacuum_statistics--1.0.sql
@@ -0,0 +1,191 @@
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION vacuum_statistics" to load this file. \quit
+
+-- schema: extvac
+CREATE SCHEMA IF NOT EXISTS extvac;
+
+
+-- In your extension's .sql install script
+CREATE OR REPLACE FUNCTION extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS void
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C
+STRICT
+PARALLEL SAFE;
+
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+CREATE FUNCTION pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+
+ OUT relid oid,
+
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+
+ OUT wraparound_failsafe_count integer,
+
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C
+STRICT
+VOLATILE;
+
+GRANT EXECUTE ON FUNCTION pg_stats_get_vacuum_tables TO PUBLIC;
+
+-- Tables view
+DROP VIEW IF EXISTS pg_stats_vacuum_tables;
+
+CREATE VIEW pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+
+ stats.wraparound_failsafe_count,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_class rel
+JOIN pg_namespace ns ON ns.oid = rel.relnamespace
+CROSS JOIN LATERAL pg_stats_get_vacuum_tables(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ rel.oid
+) AS stats
+WHERE rel.relkind = 'r';
+
+
+CREATE FUNCTION pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+
+ OUT relid oid,
+
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+
+ OUT wraparound_failsafe_count integer,
+
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C
+STRICT
+VOLATILE;
+
+GRANT EXECUTE ON FUNCTION pg_stats_get_vacuum_indexes TO PUBLIC;
+
+-- Indexes view
+DROP VIEW IF EXISTS pg_stats_vacuum_indexes;
+
+CREATE VIEW pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+
+ stats.wraparound_failsafe_count,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_class rel
+JOIN pg_namespace ns ON ns.oid = rel.relnamespace
+CROSS JOIN LATERAL pg_stats_get_vacuum_indexes(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ rel.oid
+) AS stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/vacuum_statistics.c b/vacuum_statistics.c
new file mode 100644
index 0000000..bba71f1
--- /dev/null
+++ b/vacuum_statistics.c
@@ -0,0 +1,636 @@
+#include "postgres.h"
+
+#include "pgstat.h"
+#include "fmgr.h"
+#include "miscadmin.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "storage/shmem.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/memutils.h"
+#include "common/hashfn.h"
+#include "storage/spin.h"
+#include "utils/fmgrprotos.h"
+#include "funcapi.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_class.h"
+#include "utils/lsyscache.h"
+
+/* Public module hooks */
+void _PG_init(void);
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+#define SJ_NODENAME "VacuumStatistics"
+
+/* --- GUCs --- */
+static bool evc_enabled = true;
+static int evc_max_entries = 10000;
+
+/* --- Hook chaining --- */
+static shmem_request_hook_type prev_shmem_request_hook = NULL;
+static shmem_startup_hook_type prev_shmem_startup_hook = NULL;
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook;
+
+/* --- Names --- */
+#define EVC_STATE_NAME "extvac_shared_state"
+#define EVC_HASH_NAME "extvac_hash"
+#define EVC_TRANCHE_NAME "extvac_tranche"
+
+/* --- Forward declarations --- */
+static Size evc_memsize(void);
+static void evc_shmem_request(void);
+static void evc_shmem_startup(void);
+static void evc_drop_access_hook(ObjectAccessType access,
+ Oid classId,
+ Oid objectId,
+ int subId,
+ void *arg);
+
+/* ---- Key / Entry ---- */
+
+typedef struct ExtVacKey
+{
+ Oid dboid; /* InvalidOid for shared catalogs */
+ Oid relid; /* relation OID */
+ uint8 type; /* ExtVacReportType (heap / index / …) */
+} ExtVacKey;
+
+typedef struct ExtVacEntry
+{
+ /* hash key MUST be first when using HASH_BLOBS */
+ ExtVacKey key;
+
+ /* stats payload */
+ PgStat_VacuumRelationCounts stats;
+
+ /* metadata */
+ TimestampTz first_seen;
+ TimestampTz stats_reset;
+} ExtVacEntry;
+
+typedef struct ExtVacSharedState
+{
+ LWLock lock;
+ LWLock evc_lock_hash;
+
+ dsa_handle evc_dsa_handler;
+ bool evc_changed;
+} ExtVacSharedState;
+
+static HTAB *evc_hash = NULL;
+static ExtVacSharedState *evc = NULL;
+
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params);
+
+#define ACCUMULATE_FIELD(field) dst->field += src->field;
+
+#define ACCUMULATE_SUBFIELD(substruct, field) \
+ (dst->substruct.field += src->substruct.field)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts *dst, const PgStat_CommonCounts *src)
+{
+ ACCUMULATE_FIELD(total_blks_read);
+ ACCUMULATE_FIELD(total_blks_hit);
+ ACCUMULATE_FIELD(total_blks_dirtied);
+ ACCUMULATE_FIELD(total_blks_written);
+
+ ACCUMULATE_FIELD(blks_fetched);
+ ACCUMULATE_FIELD(blks_hit);
+
+ ACCUMULATE_FIELD(wal_records);
+ ACCUMULATE_FIELD(wal_fpi);
+ ACCUMULATE_FIELD(wal_bytes);
+
+ ACCUMULATE_FIELD(blk_read_time);
+ ACCUMULATE_FIELD(blk_write_time);
+ ACCUMULATE_FIELD(delay_time);
+ ACCUMULATE_FIELD(total_time);
+
+ ACCUMULATE_FIELD(wraparound_failsafe_count);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats_relations(PgStat_VacuumRelationCounts *dst, const PgStat_VacuumRelationCounts *src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ //Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB && src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ ACCUMULATE_SUBFIELD(common, blks_fetched);
+ ACCUMULATE_SUBFIELD(common, blks_hit);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ ACCUMULATE_SUBFIELD(table, tuples_deleted);
+ ACCUMULATE_SUBFIELD(table, pages_scanned);
+ ACCUMULATE_SUBFIELD(table, pages_removed);
+ ACCUMULATE_SUBFIELD(table, vm_new_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, tuples_frozen);
+ ACCUMULATE_SUBFIELD(table, recently_dead_tuples);
+ ACCUMULATE_SUBFIELD(table, index_vacuum_count);
+ ACCUMULATE_SUBFIELD(table, missed_dead_pages);
+ ACCUMULATE_SUBFIELD(table, missed_dead_tuples);
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ ACCUMULATE_SUBFIELD(table, tuples_deleted);
+ ACCUMULATE_SUBFIELD(index, pages_deleted);
+ }
+}
+
+void
+_PG_init(void)
+{
+ /*
+ * In order to create our shared memory area, we have to be loaded via
+ * shared_preload_libraries. If not, fall out without hooking into any of
+ * the main system. (We don't throw error here because it seems useful to
+ * allow the vacuum_statistics functions to be created even when the
+ * module isn't active. The functions must protect themselves against
+ * being called then, however.)
+ */
+ if (!process_shared_preload_libraries_in_progress)
+ return;
+
+
+ /* GUCs */
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extension vacuum statistics collection.",
+ NULL,
+ &evc_enabled,
+ true,
+ PGC_SUSET, 0,
+ NULL, NULL, NULL);
+
+ DefineCustomIntVariable("vacuum_statistics.max_entries",
+ "Maximum number of hash table entries.",
+ NULL,
+ &evc_max_entries,
+ 10000, /* default */
+ 100, /* min */
+ INT_MAX / 2, /* max */
+ PGC_POSTMASTER, 0,
+ NULL, NULL, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ /* Chain shmem hooks */
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = evc_shmem_request;
+
+ prev_shmem_startup_hook = shmem_startup_hook;
+ shmem_startup_hook = evc_shmem_startup;
+
+ /* If you piggyback on pgstat vacuum report hook, chain it here */
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evc_drop_access_hook;
+}
+
+static Size
+evc_memsize(void)
+{
+ Size sz = 0;
+
+ /* shared state header */
+ sz = MAXALIGN(sizeof(ExtVacSharedState));
+
+ /* dynahash buckets + entries */
+ sz = add_size(sz,
+ hash_estimate_size(evc_max_entries, sizeof(ExtVacEntry)));
+
+ return sz;
+}
+
+static void
+evc_shmem_request(void)
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
+ /* Ask postmaster for our memory slice */
+ RequestAddinShmemSpace(evc_memsize());
+}
+
+static void
+evc_shmem_startup(void)
+{
+ HASHCTL ctl;
+ bool found;
+
+ if (prev_shmem_startup_hook)
+ prev_shmem_startup_hook();
+
+ evc = NULL;
+ evc_hash = NULL;
+
+ LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE);
+
+ /* Shared state header */
+ evc = ShmemInitStruct(EVC_STATE_NAME,
+ sizeof(ExtVacSharedState),
+ &found);
+
+ /* First time only: resolve our tranche root (optional to store) */
+ if (!found)
+ {
+ evc->evc_dsa_handler = DSM_HANDLE_INVALID;
+
+ evc->evc_changed = false;
+
+ LWLockInitialize(&evc->lock, LWLockNewTrancheId());
+ LWLockInitialize(&evc->evc_lock_hash, LWLockNewTrancheId());
+ }
+
+ /* dynahash parameters */
+ ctl.keysize = sizeof(ExtVacKey);
+ ctl.entrysize = sizeof(ExtVacEntry);
+ evc_hash = ShmemInitHash(EVC_HASH_NAME, evc_max_entries, evc_max_entries,
+ &ctl, HASH_ELEM | HASH_BLOBS);
+
+ LWLockRelease(AddinShmemInitLock);
+
+ LWLockRegisterTranche(evc->lock.tranche, EVC_TRANCHE_NAME);
+ LWLockRegisterTranche(evc->evc_lock_hash.tranche, EVC_TRANCHE_NAME);
+
+ //if (!IsUnderPostmaster && !found)
+}
+
+static ExtVacEntry *
+evc_store(Oid dboid, Oid relid, bool shared, uint8 type,
+ PgStat_VacuumRelationCounts *counts)
+{
+ bool tblOverflow;
+ HASHACTION action;
+ ExtVacEntry *e;
+ bool found = false;
+ ExtVacKey key = { .dboid = dboid, .relid = relid, .type = type };
+
+ if (!evc_enabled || evc_hash == NULL)
+ return NULL;
+
+ if (shared)
+ dboid = InvalidOid;
+
+ Assert(!LWLockHeldByMe(&evc->evc_lock_hash));
+
+ LWLockAcquire(&evc->evc_lock_hash, LW_EXCLUSIVE);
+
+ tblOverflow = hash_get_num_entries(evc_hash) < evc_max_entries ? false : true;
+
+ if (!tblOverflow)
+ action = tblOverflow ? HASH_FIND : HASH_ENTER;
+
+ e = (ExtVacEntry *) hash_search(evc_hash, &key, action, &found);
+
+ if (!found)
+ {
+ if (action == HASH_FIND)
+ {
+ /*
+ * Hash table is full. To avoid possible problems - don't try to add
+ * more, just exit
+ */
+ LWLockRelease(&evc->evc_lock_hash);
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("[Vacuum Statistics] Data storage is full. No more data can be added."),
+ errhint("Increase value of evc_max_entries on restart of the instance")));
+ return NULL;
+ }
+
+ memset(&e->stats, 0, sizeof(e->stats));
+ e->stats.type = type;
+ e->first_seen = GetCurrentTimestamp();
+ e->stats_reset = e->first_seen;
+ }
+
+ pgstat_accumulate_extvac_stats_relations(&e->stats, counts);
+
+ evc->evc_changed = true;
+ LWLockRelease(&evc->evc_lock_hash);
+
+ return e;
+}
+
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params)
+{
+ /* Call ours */
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ evc_store(dboid, tableoid, shared, params->type, params);
+
+ /* Chain to previous if any */
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+static void
+evc_entry_reset(ExtVacEntry *e)
+{
+ Assert(e != NULL);
+
+ LWLockAcquire(&evc->evc_lock_hash, LW_EXCLUSIVE);
+
+ /* wipe stats, but preserve type and key metadata */
+ memset(&e->stats, 0, sizeof(e->stats));
+ e->stats.type = e->key.type; /* relatch type */
+ e->stats_reset = GetCurrentTimestamp();
+
+ LWLockRelease(&evc->evc_lock_hash);
+}
+
+static void
+evc_reset_by_relid(Oid dboid, Oid relid, uint8 type)
+{
+ ExtVacKey key;
+ ExtVacEntry *e;
+
+ if (!evc || !evc_hash)
+ return;
+
+ key.dboid = dboid;
+ key.relid = relid;
+ key.type = type;
+
+ e = (ExtVacEntry *) hash_search(evc_hash, &key, HASH_FIND, NULL);
+ if (e)
+ evc_entry_reset(e);
+}
+
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int32 type = PG_GETARG_INT32(2);
+
+ evc_reset_by_relid(dboid, relid, (uint8) type);
+
+ PG_RETURN_VOID();
+}
+
+/*
+ * Object access hook
+ */
+static void
+evc_drop_access_hook(ObjectAccessType access,
+ Oid classId,
+ Oid objectId,
+ int subId,
+ void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ char relkind = get_rel_relkind(objectId);
+
+ if(classId == RelationRelationId && subId == 0)
+ {
+ if(relkind == RELKIND_RELATION)
+ evc_reset_by_relid(MyDatabaseId, objectId, PGSTAT_EXTVAC_TABLE);
+ else if (relkind == RELKIND_INDEX)
+ evc_reset_by_relid(MyDatabaseId, objectId, PGSTAT_EXTVAC_INDEX);
+ }
+ }
+}
+
+
+/* Number of output arguments (columns) of vacuum stats for various API versions */
+#define EXTVAC_COMMON_STAT_COLS 12 /* maximum of above */
+
+static void
+tuplestore_put_common(PgStat_CommonCounts *vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+
+ /* If you meant 12, fix the constant. Otherwise add the missing field. */
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts *vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+ memset(nulls, 0, sizeof(nulls));
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common,
+ values, nulls, &i);
+
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched -
+ vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->index.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+/*
+ * Get the vacuum statistics for the heap tables or indexes.
+ * See comment to pgpro_stats_statements() about SQL API.
+ */
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ /* Check if caller supports us returning a tuplestore */
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("vacuum statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("vacuum statistics: materialize mode required, but it is not allowed in this context")));
+
+ /* Switch to long-lived context to create the returned data structures */
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "vacuum statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ ExtVacEntry *vacuum_ext;
+
+ /* Load table statistics for specified database. */
+
+ if (OidIsValid(relid))
+ {
+ ExtVacKey key;
+
+ if (!evc || !evc_hash)
+ return (Datum) 0;
+
+ key.dboid = dbid;
+ key.relid = relid;
+ key.type = type;
+
+ LWLockAcquire(&evc->evc_lock_hash, LW_SHARED);
+
+ vacuum_ext = (ExtVacEntry *) hash_search(evc_hash, &key, HASH_FIND, NULL);
+
+ if (vacuum_ext == NULL || vacuum_ext->stats.type != type)
+ {
+ /* Table don't exists or isn't an heap relation. */
+ LWLockRelease(&evc->evc_lock_hash);
+ return (Datum) 0;
+ }
+
+ LWLockRelease(&evc->evc_lock_hash);
+
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, &vacuum_ext->stats);
+ }
+ }
+ // else if (type == PGSTAT_EXTVAC_DB)
+ // {
+ // PgStat_CommonCounts *vacuum_ext;
+
+ // vacuum_ext = fetch_dbstat_dbentry(dbid);
+
+ // if (vacuum_ext == NULL)
+ // /* Table doesn't exist or isn't a heap relation */
+ // PG_RETURN_NULL();
+
+ // Datum values[EXTVAC_COMMON_STAT_COLS];
+ // bool nulls[EXTVAC_COMMON_STAT_COLS];
+ // int i = 0;
+
+ // memset(nulls, 0, EXTVAC_COMMON_STAT_COLS * sizeof(bool));
+
+ // values[i++] = ObjectIdGetDatum(dbid);
+
+ // tuplestore_put_common(tupstore, tupdesc, vacuum_ext,
+ // &values, &nulls, &i);
+
+ // tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+
+ // return (Datum) 0;
+ // }
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+
+ PG_RETURN_NULL();
+}
+
+/*
+ * Get the vacuum statistics for the indexes.
+*/
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+
+ PG_RETURN_NULL();
+}
+
+/*
+ * Get the vacuum statistics for the databases.
+ */
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+
+ PG_RETURN_NULL();
+}
\ No newline at end of file
diff --git a/vacuum_statistics.control b/vacuum_statistics.control
new file mode 100644
index 0000000..1c72754
--- /dev/null
+++ b/vacuum_statistics.control
@@ -0,0 +1,4 @@
+comment = 'vacuum statistics'
+default_version = '1.0'
+module_pathname = '$libdir/vacuum_statistics'
+relocatable = true
\ No newline at end of file
--
2.34.1
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-09-04 16:18 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2025-09-04 16:18 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Amit Kapila <[email protected]>; pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
To be honest, I haven’t provided extensions for the PostgreSQL [0] to
hackers yet, nor have I encountered this situation in general. Just in
case, I created an open repository on GitHub with the code and added a
description in the README.
[0] https://github.com/Alena0704/vacuum_statistics#
On 04.09.2025 18:49, Alena Rybakina wrote:
> Hi, all!
>
> On 02.06.2025 19:50, Alena Rybakina wrote:
>>
>> On 02.06.2025 19:25, Alexander Korotkov wrote:
>>> On Tue, May 13, 2025 at 12:49 PM Alena Rybakina
>>> <[email protected]> wrote:
>>>> On 12.05.2025 08:30, Amit Kapila wrote:
>>>>> On Fri, May 9, 2025 at 5:34 PM Alena Rybakina
>>>>> <[email protected]> wrote:
>>>>>> I did a rebase and finished the part with storing statistics
>>>>>> separately from the relation statistics - now it is possible to
>>>>>> disable the collection of statistics for relationsh using gucs and
>>>>>> this allows us to solve the problem with the memory consumed.
>>>>>>
>>>>> I think this patch is trying to collect data similar to what we do
>>>>> for
>>>>> pg_stat_statements for SQL statements. So, can't we follow a similar
>>>>> idea such that these additional statistics will be collected once
>>>>> some
>>>>> external module like pg_stat_statements is enabled? That module
>>>>> should
>>>>> be responsible for accumulating and resetting the data, so we won't
>>>>> have this memory consumption issue.
>>>> The idea is good, it will require one hook for the
>>>> pgstat_report_vacuum
>>>> function, the extvac_stats_start and extvac_stats_end functions can be
>>>> run if the extension is loaded, so as not to add more hooks.
>>> +1
>>> Nice idea of a hook. Given the volume of the patch, it might be a
>>> good idea to keep this as an extension.
>> Okay, I'll realize it and apply the patch)
>>>
>>>> But I see a problem here with tracking deleted objects for which
>>>> statistics are no longer needed. There are two solutions to this and I
>>>> don't like both of them, to be honest.
>>>> The first way is to add a background process that will go through the
>>>> table with saved statistics and check whether the relation or the
>>>> database are relevant now or not and if not, then
>>>> delete the vacuum statistics information for it. This may be
>>>> resource-intensive. The second way is to add hooks for deleting the
>>>> database and relationships (functions dropdb, index_drop,
>>>> heap_drop_with_catalog).
>>> Can we workaround this with object_access_hook?
>>
>> I think this could fix the problem. For the OAT-DROP access type, we
>> can call a function to reset the vacuum statistics for relations that
>> are about to be dropped.
>>
>> At the moment, I don’t see any limitations to using this approach.
>>
> I’ve prepared the first working version of the extension.
>
> I haven’t yet implemented writing the statistics to a file and
> reloading them into a hash table and shared memory at instance
> startup, and I also haven’t implemented a proper output for
> database-level statistics yet.
>
> I structured the extension as follows: statistics are stored in a hash
> table keyed by a composite key - database OID, relation OID, and
> object type (index, table, or database). When VACUUM or a worker
> processes a table or index, an exclusive lock is taken to update the
> corresponding record; a shared lock is taken when reading the
> statistics. For database-level output, I plan to compute the totals by
> summing table and index statistics on demand.
>
> To optimize that, I plan to keep entries in the hash table ordered by
> database OID. When accessing the first element by the partial key
> (database OID), I’ll scan forward and aggregate until the partitial
> database key changes.
>
> Right now this requires adding the extension to
> `shared_preload_libraries`. I haven’t found a way to avoid that
> because of shared-memory setup, and I’m not sure it’s even possible.
>
> I’m also unsure whether it’s better to store the statistics in the
> cumulative statistics system (as done here) or entirely inside the
> extension. Note that the code added to the core to support the
> extension executes regardless of whether the extension is enabled.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-09-15 20:46 Ilia Evdokimov <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Ilia Evdokimov @ 2025-09-15 20:46 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
Hi Alena,
Thanks for the work you’ve done.
On 01.09.2025 22:13, Alena Rybakina wrote:
> I've rebased the patches to the current HEAD.
Right now there is a bug: when I run a simple
SELECT * FROM pg_stat_vacuum_database;
psql crashes.
The root cause is an incorrect zeroing of a local variable:
PgStat_VacuumDBCounts allzero;
- memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&allzero, 0, sizeof(PgStat_VacuumDBCounts));
--
Best regards,
Ilia Evdokimov,
Tantor Labs LLC,
https://tantorlabs.com
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-09-25 00:03 Bharath Rupireddy <[email protected]>
parent: Amit Kapila <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Bharath Rupireddy @ 2025-09-25 00:03 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Alena Rybakina <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
Hi,
On Mon, May 12, 2025 at 5:30 AM Amit Kapila <[email protected]> wrote:
>
> On Fri, May 9, 2025 at 5:34 PM Alena Rybakina <[email protected]> wrote:
> >
> > I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and
> > this allows us to solve the problem with the memory consumed.
> >
>
> I think this patch is trying to collect data similar to what we do for
> pg_stat_statements for SQL statements. So, can't we follow a similar
> idea such that these additional statistics will be collected once some
> external module like pg_stat_statements is enabled? That module should
> be responsible for accumulating and resetting the data, so we won't
> have this memory consumption issue.
>
> BTW, how will these new statistics be used to autotune a vacuum? And
> do we need all the statistics proposed by this patch?
Thanks for working on this. I agree with the general idea of having
minimal changes to the core. I think a simple approach would be to
have a hook in heap_vacuum_rel at the end, where vacuum stats are
prepared in a buffer for emitting LOG messages. External modules can
then handle storing, rotating, interpreting, aggregating (per
relation/per database), and exposing the stats to end-users via SQL.
The core can define a common data structure, fill it, and send it to
external modules. I haven't had a chance to read the whole thread or
review the patches; I'm sure this has been discussed.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-09-25 13:53 Alena Rybakina <[email protected]>
parent: Ilia Evdokimov <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2025-09-25 13:53 UTC (permalink / raw)
To: Ilia Evdokimov <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>; pgsql-hackers
Hi! Thank you for reviewing it.
My email hasn’t been working properly recently, so I missed your letter
— sorry about that.
Yes, I completely agree with your proposed fix and have included it in
the updated version of the patch.
Just to remind you, this version stores vacuum statistics separately in
the cumulative system due to the issue with disabling the gathering of
vacuum statistics.
The patch had some conflicts with the current master branch, but I’ve
resolved them and now it applies cleanly without conflicts.
On 15.09.2025 23:46, Ilia Evdokimov wrote:
>
> Right now there is a bug: when I run a simple
>
> SELECT * FROM pg_stat_vacuum_database;
>
> psql crashes.
>
> The root cause is an incorrect zeroing of a local variable:
> PgStat_VacuumDBCounts allzero;
> - memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
> + memset(&allzero, 0, sizeof(PgStat_VacuumDBCounts));
>
> --
> Best regards,
> Ilia Evdokimov,
> Tantor Labs LLC,
> https://tantorlabs.com
>
>
>
>
>
Attachments:
[text/x-patch] v25-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (71.5K, 2-v25-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From d51643eebf73cffd4cce82a04bf7db62d9007afb Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 25 Sep 2025 16:20:08 +0300
Subject: [PATCH 1/5] Machinery for grabbing an extended vacuum statistics on
table relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen is a number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible is a number of pages that are marked as all-visible in vm during
vacuum.
wraparound_failsafe_count is a number of times when the vacuum starts urgent cleanup
to prevent wraparound problem which is critical for the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 150 +++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 52 +++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 46 +++-
src/backend/utils/adt/pgstatfuncs.c | 147 ++++++++++++
src/backend/utils/error/elog.c | 13 +
src/backend/utils/misc/guc_parameters.dat | 8 +-
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 80 +++++-
src/include/utils/elog.h | 1 +
.../vacuum-extending-in-repetable-read.out | 53 ++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 53 ++++
src/test/regress/expected/rules.out | 44 +++-
.../expected/vacuum_tables_statistics.out | 227 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 183 ++++++++++++++
22 files changed, 1093 insertions(+), 17 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 981d9380a92..437e5f581ae 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -289,6 +289,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -407,6 +408,8 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} LVRelState;
@@ -418,6 +421,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -474,6 +489,106 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
@@ -632,6 +747,13 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
char **indnames = NULL;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -651,7 +773,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -668,6 +790,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -776,6 +899,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->aggressive = vacuum_get_cutoffs(rel, params, &vacrel->cutoffs);
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ vacrel->wraparound_failsafe_count = 0;
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
@@ -924,6 +1048,26 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ }
+
/*
* Report results to the cumulative stats system, too.
*
@@ -939,7 +1083,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime);
+ starttime,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -2967,6 +3112,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[2] = {0, 0};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count ++;
/*
* Abandon use of a buffer access strategy to allow use of all of
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 7306c16f05c..a881c33b75a 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index c77fa0234bb..44937ca37fc 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -716,7 +716,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_vacuum_time(C.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1420,3 +1422,51 @@ REVOKE ALL ON pg_aios FROM PUBLIC;
GRANT SELECT ON pg_aios TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_aios() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.vm_new_frozen_pages AS vm_new_frozen_pages,
+ stats.vm_new_visible_pages AS vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ stats.missed_dead_pages AS missed_dead_pages,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.recently_dead_tuples AS recently_dead_tuples,
+ stats.missed_dead_tuples AS missed_dead_tuples,
+
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 733ef40ae7c..d8776ff1901 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -116,6 +116,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2533,6 +2536,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 0feea1d30ec..2b55d9b7c0e 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1054,6 +1054,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 73c2ced3f4e..400fafe921b 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = true;
/* ----------
* state shared with pgstat_*.c
@@ -265,7 +265,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -883,7 +882,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -949,7 +947,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1065,7 +1063,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1116,7 +1114,7 @@ pgstat_prep_snapshot(void)
}
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 69df741cbf6..e023926ff05 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -209,7 +211,7 @@ pgstat_drop_relation(Relation rel)
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime)
+ TimestampTz starttime, ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -235,6 +237,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -881,6 +885,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -1004,3 +1011,40 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
+ dst->vm_new_visible_pages += src->vm_new_visible_pages;
+ dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->missed_dead_pages += src->missed_dead_pages;
+ dst->missed_dead_tuples += src->missed_dead_tuples;
+
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..ee461ea378d 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2260,3 +2266,144 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index b7b9692f8c8..f0ecf86e514 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1627,6 +1627,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 6bc6be13d2a..a67887deffd 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -506,8 +506,14 @@
},
{ name => 'track_wal_io_timing', type => 'bool', context => 'PGC_SUSET', group => 'STATS_CUMULATIVE',
+ short_desc => 'Collects vacuum statistics for vacuum activity.',
+ variable => 'pgstat_track_vacuum_statistics',
+ boot_val => 'false',
+},
+
+{ name => 'track_vacuum_statistics', type => 'bool', context => 'PGC_SUSET', group => 'STATS_CUMULATIVE',
short_desc => 'Collects timing statistics for WAL I/O activity.',
- variable => 'track_wal_io_timing',
+ variable => 'track_vacuum_statistics',
boot_val => 'false',
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index c36fcb9ab61..c1f8d3f0edf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -662,6 +662,7 @@
#track_wal_io_timing = off
#track_functions = none # none, pl, all
#stats_fetch_consistency = cache # cache, none, snapshot
+#track_vacuum_statistics = off
# - Monitoring -
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 01eba3b5a19..0f41f9d0658 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12588,4 +12588,22 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int4,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 14eeccbd718..4d05e1a0fac 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -327,6 +327,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f402b17295c..9e37b96de92 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,6 +111,53 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* the number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -153,6 +200,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -211,7 +268,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB7
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB8
typedef struct PgStat_ArchiverStats
{
@@ -375,6 +432,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -453,6 +512,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autovacuum_time;
PgStat_Counter total_analyze_time;
PgStat_Counter total_autoanalyze_time;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -660,7 +724,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime);
+ TimestampTz starttime, ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -711,6 +775,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -799,6 +874,7 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
/*
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 675f4f5f469..356dadd6b0a 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrcode(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..87f7e40b4a6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 5afae33d370..645909b1969 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -99,6 +99,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5893d89573d
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,53 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..349e7deba01 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1833,7 +1833,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_vacuum_time(c.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2232,7 +2234,9 @@ pg_stat_sys_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2284,9 +2288,43 @@ pg_stat_user_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.missed_dead_pages,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.missed_dead_tuples,
+ stats.wraparound_failsafe,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..b5ea9c9ab1e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,227 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+ relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_scanned | pages_removed | vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | missed_dead_pages | tuples_deleted | tuples_frozen | recently_dead_tuples | missed_dead_tuples | index_vacuum_count | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+---------------+---------------------+----------------------+-----------------------------+-------------------+----------------+---------------+----------------------+--------------------+--------------------+-------------+---------+-----------+---------------+----------------+------------+------------
+ vestat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
+(1 row)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
+---------------------+----------------------+----------------------+-----------------------+-----------------------------
+ 0 | 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | f | f
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index fbffc67ae60..cd779ab8eca 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,3 +140,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..5bc34bec64b
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+DROP TABLE vestat CASCADE;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v25-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (54.6K, 3-v25-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 604e852929d7b9c1bf7fe3d58b8bbc0db3e12caa Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 25 Sep 2025 16:22:38 +0300
Subject: [PATCH 2/5] Machinery for grabbing an extended vacuum statistics on
index relations.
They are gathered separatelly from table statistics.
As for tables, we gather vacuum shared buffers statistics for index relations like
value of total_blks_hit, total_blks_read, total_blks_dirtied, wal statistics, io time
during flushing buffer pages to disk, delay and total time.
Due to the fact that such statistics are common as for tables, as for indexes we
set them in the union ExtVacReport structure. We only added some determination 'type'
field to highlight what kind belong to these statistics: PGSTAT_EXTVAC_TABLE or
PGSTAT_EXTVAC_INDEX. Generally, PGSTAT_EXTVAC_INVALID type leads to wrong code process.
Some statistics belong only one type of both tables or indexes. So, we added substructures
sych table and index inside ExtVacReport structure.
Therefore, we gather only for tables such statistics like number of scanned, removed pages,
their charecteristics according VM (all-visible and frozen). In addition, for tables we
gather number frozen, deleted and recently dead tuples and how many times vacuum processed
indexes for tables.
Controversally for indexes we gather number of deleted pages and deleted tuples only.
As for tables, deleted pages and deleted tuples reflect the overall performance of the vacuum
for the index relationship.
Since the vacuum cleans up references to tuple indexes before cleaning up table tuples,
which adds some complexity to the vacuum process, namely the vacuum switches from cleaning up
a table to its indexes and back during its operation, we need to save the vacuum statistics
collected for the heap before it starts cleaning up the indexes.
That's why it's necessary to track the vacuum statistics for the heap several times during
the vacuum procedure. To avoid sending the statistics to the Cumulative Statistics System
several times, we save these statistics in the LVRelState structure and only after vacuum
finishes cleaning up the heap, it sends them to the Cumulative Statistics System.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 268 ++++++++++++++----
src/backend/catalog/system_views.sql | 32 +++
src/backend/commands/vacuumparallel.c | 14 +
src/backend/utils/activity/pgstat.c | 4 +
src/backend/utils/activity/pgstat_relation.c | 48 +++-
src/backend/utils/adt/pgstatfuncs.c | 133 ++++++++-
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 25 ++
src/include/pgstat.h | 58 +++-
.../vacuum-extending-in-repetable-read.out | 4 +-
src/test/regress/expected/rules.out | 22 ++
.../expected/vacuum_index_statistics.out | 183 ++++++++++++
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 151 ++++++++++
14 files changed, 863 insertions(+), 89 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 437e5f581ae..07ad337dc82 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -290,6 +290,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -410,6 +411,8 @@ typedef struct LVRelState
BlockNumber eager_scan_remaining_fails;
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReport extVacReportIdx;
} LVRelState;
@@ -421,19 +424,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-/*
- * Counters and usage data for extended stats tracking.
- */
-typedef struct LVExtStatCounters
-{
- TimestampTz starttime;
- WalUsage walusage;
- BufferUsage bufusage;
- double VacuumDelayTime;
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-} LVExtStatCounters;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static void heap_vacuum_eager_scan_setup(LVRelState *vacrel,
@@ -555,27 +545,25 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
endtime = GetCurrentTimestamp();
TimestampDifference(counters->starttime, endtime, &secs, &usecs);
- memset(report, 0, sizeof(ExtVacReport));
-
/*
* Fill additional statistics on a vacuum processing operation.
*/
- report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
- report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
- report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
- report->total_blks_written = bufusage.shared_blks_written;
+ report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written += bufusage.shared_blks_written;
- report->wal_records = walusage.wal_records;
- report->wal_fpi = walusage.wal_fpi;
- report->wal_bytes = walusage.wal_bytes;
+ report->wal_records += walusage.wal_records;
+ report->wal_fpi += walusage.wal_fpi;
+ report->wal_bytes += walusage.wal_bytes;
- report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
- report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
- report->total_time = secs * 1000. + usecs / 1000.;
+ report->total_time += secs * 1000. + usecs / 1000.;
if (!rel->pgstat_info || !pgstat_track_counts)
/*
@@ -584,12 +572,131 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
*/
return;
- report->blks_fetched =
+ report->blks_fetched +=
rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
- report->blks_hit =
+ report->blks_hit +=
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Set initial values for common heap and index statistics*/
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ memset(report, 0, sizeof(ExtVacReport));
+
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
+ extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
+ extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
+ extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
+ extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
+ extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
+ extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
+ extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
+ extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+
+ extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
+ extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+
+}
+
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacIdxStats)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
+ vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
+ vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
+ vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
+ vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
+ vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
+ vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
+ vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
+ vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
+ vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
+
+ vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -752,7 +859,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ExtVacReport allzero;
/* Initialize vacuum statistics */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
extVacReport = allzero;
verbose = (params.options & VACOPT_VERBOSE) != 0;
@@ -799,6 +906,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
vac_open_indexes(vacrel->rel, RowExclusiveLock, &vacrel->nindexes,
@@ -1051,23 +1160,6 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Make generic extended vacuum stats report */
extvac_stats_end(rel, &extVacCounters, &extVacReport);
- if(pgstat_track_vacuum_statistics)
- {
- /* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
- extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
- extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
- extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
- extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- }
-
/*
* Report results to the cumulative stats system, too.
*
@@ -1078,13 +1170,34 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(RelationGetRelid(rel),
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
+ vacrel->missed_dead_tuples,
starttime,
&extVacReport);
+
+ }
+ else
+ {
+ pgstat_report_vacuum(RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+ }
+
pgstat_progress_end_command();
if (instrument)
@@ -2782,10 +2895,20 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
* that parallel VACUUM only gets the precheck and this postcheck.
@@ -3195,10 +3318,20 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
estimated_count);
+
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
/* Reset the progress counters */
@@ -3224,6 +3357,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3242,6 +3380,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3250,6 +3389,19 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3274,6 +3426,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3293,12 +3450,25 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 44937ca37fc..4bbd1499bb8 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1470,3 +1470,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 2b55d9b7c0e..65de45a4447 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,15 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 400fafe921b..fd2c5e15369 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1159,6 +1159,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index e023926ff05..c6194584b35 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -1016,6 +1016,9 @@ static void
pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
bool accumulate_reltype_specific_info)
{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
dst->total_blks_read += src->total_blks_read;
dst->total_blks_hit += src->total_blks_hit;
dst->total_blks_dirtied += src->total_blks_dirtied;
@@ -1031,20 +1034,35 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
- dst->vm_new_visible_pages += src->vm_new_visible_pages;
- dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->missed_dead_pages += src->missed_dead_pages;
- dst->missed_dead_tuples += src->missed_dead_tuples;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->tuples_deleted += src->tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index ee461ea378d..defe1990e11 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2374,18 +2374,19 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2404,6 +2405,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0f41f9d0658..2a8b6b2c1b8 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12606,4 +12606,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 4d05e1a0fac..dcc542750b8 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -295,6 +296,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -408,4 +429,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 9e37b96de92..1f1402d4179 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -111,11 +111,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -144,18 +152,44 @@ typedef struct ExtVacReport
double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
double total_time; /* total time of a vacuum operation, in msec */
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
- int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
- int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
- int64 missed_dead_pages; /* pages with missed dead tuples */
int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
- int64 index_vacuum_count; /* the number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 87f7e40b4a6..6d960423912 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -34,7 +34,7 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
(1 row)
step s1_commit: COMMIT;
@@ -48,6 +48,6 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
(1 row)
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 349e7deba01..fdd5341bdfc 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2293,6 +2293,28 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schemaname,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
rel.relname,
stats.relid,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..e00a0fc683c
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+ relid | schemaname | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+------------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+(0 rows)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index cd779ab8eca..4eb03353104 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -144,4 +144,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..ae146e1d23f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,151 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
--
2.34.1
[text/x-patch] v25-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (30.8K, 4-v25-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 1aa51ec901cbf46e53137fa099b5474ebca5b614 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 1 Sep 2025 21:43:33 +0300
Subject: [PATCH 3/5] Machinery for grabbing an extended vacuum statistics on
databases.
Database vacuum statistics information is the collected general
vacuum statistics indexes and tables owned by the databases, which
they belong to.
In addition to the fact that there are far fewer databases in a system
than relations, vacuum statistics for a database contain fewer statistics
than relations, but they are enough to indicate that something may be
wrong in the system and prompt the administrator to enable extended
monitoring for relations.
So, buffer, wal, statistics of I/O time of read and writen blocks
statistics will be observed because they are collected for both
tables, indexes. In addition, we show the number of errors caught
during operation of the vacuum only for the error level.
wraparound_failsafe_count is a number of times when the vacuum starts
urgent cleanup to prevent wraparound problem which is critical for
the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 17 ++-
src/backend/catalog/system_views.sql | 27 ++++-
src/backend/utils/activity/pgstat.c | 2 +-
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 46 +++++++-
src/backend/utils/adt/pgstatfuncs.c | 100 +++++++++++++++++-
src/include/catalog/pg_proc.dat | 13 ++-
src/include/pgstat.h | 5 +-
.../vacuum-extending-in-repetable-read.spec | 6 ++
src/test/regress/expected/rules.out | 17 +++
.../expected/vacuum_index_statistics.out | 16 +--
...ut => vacuum_tables_and_db_statistics.out} | 87 +++++++++++++--
src/test/regress/parallel_schedule | 2 +-
.../regress/sql/vacuum_index_statistics.sql | 6 +-
...ql => vacuum_tables_and_db_statistics.sql} | 69 +++++++++++-
15 files changed, 380 insertions(+), 34 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (82%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (81%)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 07ad337dc82..aac65933e1a 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -659,7 +659,7 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
@@ -4081,6 +4081,9 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4096,6 +4099,9 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -4111,16 +4117,25 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 4bbd1499bb8..1c5e351a672 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1501,4 +1501,29 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.errors AS errors
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index fd2c5e15369..6cb9077a27f 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-bool pgstat_track_vacuum_statistics = true;
+bool pgstat_track_vacuum_statistics = false;
/* ----------
* state shared with pgstat_*.c
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index b31f20d41bc..65207d30378 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -485,6 +485,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index c6194584b35..bf7ab345be0 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -205,6 +205,38 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.type = m_type;
+ pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.errors++;
+ dbentry->vacuum_ext.type = m_type;
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
@@ -216,6 +248,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -274,6 +307,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
}
/*
@@ -1030,6 +1073,8 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->blk_write_time += src->blk_write_time;
dst->delay_time += src->delay_time;
dst->total_time += src->total_time;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->errors += src->errors;
if (!accumulate_reltype_specific_info)
return;
@@ -1057,7 +1102,6 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->table.index_vacuum_count += src->table.index_vacuum_count;
dst->table.missed_dead_pages += src->table.missed_dead_pages;
dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
}
else if (dst->type == PGSTAT_EXTVAC_INDEX)
{
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index defe1990e11..1c39ada2c3e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2385,7 +2385,7 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
@@ -2515,6 +2515,104 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2a8b6b2c1b8..49c3c16ce5b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12607,12 +12607,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,wraparound_failsafe,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1f1402d4179..6952f833d45 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -154,6 +154,9 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
+ int32 errors;
+ int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+
ExtVacReportType type; /* heap, index, etc. */
/* ----------
@@ -183,7 +186,6 @@ typedef struct ExtVacReport
int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
} table;
struct
{
@@ -762,6 +764,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5893d89573d..cfec3159580 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -18,6 +18,9 @@ teardown
}
session s1
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s1_begin_repeatable_read {
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
select count(ival) from test_vacuum_stat_isolation where id>900;
@@ -25,6 +28,9 @@ step s1_begin_repeatable_read {
step s1_commit { COMMIT; }
session s2
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index fdd5341bdfc..14f19e5bcbe 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2293,6 +2293,23 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schemaname,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index e00a0fc683c..9e5d33342c9 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -16,8 +16,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -33,12 +37,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -181,3 +180,4 @@ WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
(1 row)
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 82%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index b5ea9c9ab1e..0300e7b6276 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,7 +6,6 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
--------------
@@ -16,8 +15,12 @@ SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
+ track_vacuum_statistics
+-------------------------
+ off
+(1 row)
+
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
@@ -37,12 +40,12 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
- track_vacuum_statistics
--------------------------
- on
-(1 row)
-
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -225,3 +228,69 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
(1 row)
DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4eb03353104..798692cf21a 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -145,4 +145,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index ae146e1d23f..9b7e645187d 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -14,8 +14,7 @@ SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -33,7 +32,7 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+SET track_vacuum_statistics TO 'on';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -149,3 +148,4 @@ FROM pg_stat_vacuum_indexes vt, pg_class c
WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
DROP TABLE vestat;
+RESET track_vacuum_statistics;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 81%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index 5bc34bec64b..ca7dbde9387 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,15 +7,13 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
\set sample_size 10000
-- not enabled by default, but we want to test it...
SET track_functions TO 'all';
--- Test that vacuum statistics will be empty when parameter is off.
-SET track_vacuum_statistics TO 'off';
+SHOW track_vacuum_statistics; -- must be off
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -36,7 +34,13 @@ WHERE vt.relname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
-SHOW track_vacuum_statistics; -- must be on
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+SET track_vacuum_statistics TO on;
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -180,4 +184,59 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+DROP TABLE vestat CASCADE;
+
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+SET track_vacuum_statistics TO on;
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SET track_vacuum_statistics TO on;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
+RESET track_vacuum_statistics;
\ No newline at end of file
--
2.34.1
[text/x-patch] v25-0004-Vacuum-statistics-have-been-separated-from-regular-r.patch (93.8K, 5-v25-0004-Vacuum-statistics-have-been-separated-from-regular-r.patch)
download | inline diff:
From 0084f46c8b4a8bfe8d81f54747c305e48753a920 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 1 Sep 2025 21:44:59 +0300
Subject: [PATCH 4/5] Vacuum statistics have been separated from regular
relation and database statistics to reduce memory usage. Dedicated
PGSTAT_KIND_VACUUM_RELATION and PGSTAT_KIND_VACUUM_DB entries were added to
the stats collector to efficiently allocate memory for vacuum-specific
metrics, which require significantly more space per relation.
---
src/backend/access/heap/vacuumlazy.c | 196 ++++++------
src/backend/catalog/heap.c | 1 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/system_views.sql | 178 +++++------
src/backend/commands/dbcommands.c | 1 +
src/backend/commands/vacuumparallel.c | 14 +-
src/backend/utils/activity/Makefile | 1 +
src/backend/utils/activity/pgstat.c | 28 ++
src/backend/utils/activity/pgstat_database.c | 10 +-
src/backend/utils/activity/pgstat_relation.c | 111 +------
src/backend/utils/activity/pgstat_vacuum.c | 215 +++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 285 +++++-------------
src/include/commands/vacuum.h | 2 +-
src/include/pgstat.h | 200 ++++++------
src/include/utils/pgstat_internal.h | 15 +
src/include/utils/pgstat_kind.h | 4 +-
src/test/regress/expected/rules.out | 146 ++++-----
.../expected/vacuum_index_statistics.out | 82 ++---
.../regress/sql/vacuum_index_statistics.sql | 48 +--
19 files changed, 805 insertions(+), 733 deletions(-)
create mode 100644 src/backend/utils/activity/pgstat_vacuum.c
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index aac65933e1a..57cd144f77a 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -412,7 +412,7 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
- ExtVacReport extVacReportIdx;
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -524,7 +524,7 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters *counters,
- ExtVacReport *report)
+ PgStat_CommonCounts *report)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -585,6 +585,8 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
if(!pgstat_track_vacuum_statistics)
return;
+ memset(&counters->common, 0, sizeof(LVExtStatCounters));
+
/* Set initial values for common heap and index statistics*/
extvac_stats_start(rel, &counters->common);
counters->pages_deleted = counters->tuples_removed = 0;
@@ -602,11 +604,13 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
void
extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report)
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report)
{
- memset(report, 0, sizeof(ExtVacReport));
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ extvac_stats_end(rel, &counters->common, &report->common);
- extvac_stats_end(rel, &counters->common, report);
report->type = PGSTAT_EXTVAC_INDEX;
if (stats != NULL)
@@ -617,7 +621,7 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
*/
/* Fill index-specific extended stats fields */
- report->tuples_deleted =
+ report->common.tuples_deleted =
stats->tuples_removed - counters->tuples_removed;
report->index.pages_deleted =
stats->pages_deleted - counters->pages_deleted;
@@ -640,7 +644,7 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
* procudure.
*/
static void
-accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacStats)
{
if (!pgstat_track_vacuum_statistics)
return;
@@ -652,49 +656,49 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacStats)
extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
- extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
- extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
- extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
- extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
- extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
- extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
- extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
- extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
- extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
- extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
}
static void
-accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport *extVacIdxStats)
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacIdxStats)
{
if (!pgstat_track_vacuum_statistics)
return;
/* Fill heap-specific extended stats fields */
- vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
- vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
- vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
- vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
- vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
- vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
- vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
- vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
- vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
- vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
-
- vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
}
@@ -855,12 +859,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ErrorContextCallback errcallback;
char **indnames = NULL;
LVExtStatCounters extVacCounters;
- ExtVacReport extVacReport;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts extVacReport;
/* Initialize vacuum statistics */
- memset(&extVacReport, 0, sizeof(ExtVacReport));
- extVacReport = allzero;
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -906,7 +908,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
- memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+ memset(&vacrel->extVacReportIdx, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
@@ -1158,7 +1161,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
&frozenxid_updated, &minmulti_updated, false);
/* Make generic extended vacuum stats report */
- extvac_stats_end(rel, &extVacCounters, &extVacReport);
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
/*
* Report results to the cumulative stats system, too.
@@ -1170,33 +1173,20 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- if(pgstat_track_vacuum_statistics)
- {
- /* Make generic extended vacuum stats report and
- * fill heap-specific extended stats fields.
- */
- extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
- accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime,
- &extVacReport);
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
- }
- else
- {
- pgstat_report_vacuum(RelationGetRelid(rel),
+ pgstat_report_vacuum_extstats(vacrel->reloid, rel->rd_rel->relisshared, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(rel),
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime,
- NULL);
- }
+ starttime);
pgstat_progress_end_command();
@@ -2896,9 +2886,9 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -2906,7 +2896,7 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
- extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
/*
@@ -3319,9 +3309,9 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -3330,7 +3320,7 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
vacrel->num_index_scans,
estimated_count);
- extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
@@ -3358,7 +3348,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3389,18 +3382,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- if (!ParallelVacuumIsActive(vacrel))
- accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ pgstat_report_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -3427,7 +3415,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3457,17 +3448,13 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_cleanup_one_index(&ivinfo, istat);
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
- if (!ParallelVacuumIsActive(vacrel))
- accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
-
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -4067,6 +4054,27 @@ update_relstats_all_indexes(LVRelState *vacrel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+static void
+pgstat_report_vacuum_error()
+{
+ PgStat_VacuumDBCounts *vacuum_dbentry;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ vacuum_dbentry = pgstat_fetch_stat_vacuum_dbentry(MyDatabaseId);
+
+ if(vacuum_dbentry == NULL)
+ return;
+ vacuum_dbentry->errors++;
+}
+
/*
* Error context callback for errors occurring during vacuum. The error
* context messages for index phases should match the messages set in parallel
@@ -4082,7 +4090,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -4100,7 +4108,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
{
@@ -4118,7 +4126,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+ pgstat_report_vacuum_error();
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
@@ -4126,7 +4134,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+ pgstat_report_vacuum_error();
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
@@ -4134,7 +4142,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_TRUNCATE:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+ pgstat_report_vacuum_error();
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index fd6537567ea..f82f11490ff 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1883,6 +1883,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5d9db167e59..7def7c13b15 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2327,6 +2327,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c5e351a672..28bbf11cf97 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1430,100 +1430,104 @@ GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
--
CREATE VIEW pg_stat_vacuum_tables AS
-SELECT
- ns.nspname AS schemaname,
- rel.relname AS relname,
- stats.relid as relid,
-
- stats.total_blks_read AS total_blks_read,
- stats.total_blks_hit AS total_blks_hit,
- stats.total_blks_dirtied AS total_blks_dirtied,
- stats.total_blks_written AS total_blks_written,
-
- stats.rel_blks_read AS rel_blks_read,
- stats.rel_blks_hit AS rel_blks_hit,
-
- stats.pages_scanned AS pages_scanned,
- stats.pages_removed AS pages_removed,
- stats.vm_new_frozen_pages AS vm_new_frozen_pages,
- stats.vm_new_visible_pages AS vm_new_visible_pages,
- stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
- stats.missed_dead_pages AS missed_dead_pages,
- stats.tuples_deleted AS tuples_deleted,
- stats.tuples_frozen AS tuples_frozen,
- stats.recently_dead_tuples AS recently_dead_tuples,
- stats.missed_dead_tuples AS missed_dead_tuples,
-
- stats.wraparound_failsafe AS wraparound_failsafe,
- stats.index_vacuum_count AS index_vacuum_count,
- stats.wal_records AS wal_records,
- stats.wal_fpi AS wal_fpi,
- stats.wal_bytes AS wal_bytes,
-
- stats.blk_read_time AS blk_read_time,
- stats.blk_write_time AS blk_write_time,
-
- stats.delay_time AS delay_time,
- stats.total_time AS total_time
-
-FROM pg_class rel
- JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
- LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
-WHERE rel.relkind = 'r';
+ SELECT
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ S.relid as relid,
+
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
+
+ S.pages_scanned AS pages_scanned,
+ S.pages_removed AS pages_removed,
+ S.vm_new_frozen_pages AS vm_new_frozen_pages,
+ S.vm_new_visible_pages AS vm_new_visible_pages,
+ S.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ S.missed_dead_pages AS missed_dead_pages,
+ S.tuples_deleted AS tuples_deleted,
+ S.tuples_frozen AS tuples_frozen,
+ S.recently_dead_tuples AS recently_dead_tuples,
+ S.missed_dead_tuples AS missed_dead_tuples,
+
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.index_vacuum_count AS index_vacuum_count,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+
+ FROM pg_class C JOIN
+ pg_namespace N ON N.oid = C.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(C.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
CREATE VIEW pg_stat_vacuum_indexes AS
-SELECT
- rel.oid as relid,
- ns.nspname AS schemaname,
- rel.relname AS relname,
+ SELECT
+ C.oid AS relid,
+ I.oid AS indexrelid,
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ I.relname AS indexrelname,
- total_blks_read AS total_blks_read,
- total_blks_hit AS total_blks_hit,
- total_blks_dirtied AS total_blks_dirtied,
- total_blks_written AS total_blks_written,
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
- rel_blks_read AS rel_blks_read,
- rel_blks_hit AS rel_blks_hit,
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
- pages_deleted AS pages_deleted,
- tuples_deleted AS tuples_deleted,
+ S.pages_deleted AS pages_deleted,
+ S.tuples_deleted AS tuples_deleted,
- wal_records AS wal_records,
- wal_fpi AS wal_fpi,
- wal_bytes AS wal_bytes,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
- blk_read_time AS blk_read_time,
- blk_write_time AS blk_write_time,
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
- delay_time AS delay_time,
- total_time AS total_time
-FROM
- pg_class rel
- JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
- LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+ FROM
+ pg_class C JOIN
+ pg_index X ON C.oid = X.indrelid JOIN
+ pg_class I ON I.oid = X.indexrelid
+ LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace),
+ LATERAL pg_stat_get_vacuum_indexes(I.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
CREATE VIEW pg_stat_vacuum_database AS
-SELECT
- db.oid as dboid,
- db.datname AS dbname,
-
- stats.db_blks_read AS db_blks_read,
- stats.db_blks_hit AS db_blks_hit,
- stats.total_blks_dirtied AS total_blks_dirtied,
- stats.total_blks_written AS total_blks_written,
-
- stats.wal_records AS wal_records,
- stats.wal_fpi AS wal_fpi,
- stats.wal_bytes AS wal_bytes,
-
- stats.blk_read_time AS blk_read_time,
- stats.blk_write_time AS blk_write_time,
-
- stats.delay_time AS delay_time,
- stats.total_time AS total_time,
- stats.wraparound_failsafe AS wraparound_failsafe,
- stats.errors AS errors
-FROM
- pg_database db,
- LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
+ SELECT
+ D.oid as dboid,
+ D.datname AS dbname,
+
+ S.db_blks_read AS db_blks_read,
+ S.db_blks_hit AS db_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time,
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.errors AS errors
+ FROM
+ pg_database D,
+ LATERAL pg_stat_get_vacuum_database(D.oid) S;
\ No newline at end of file
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 2793fd83771..25ede1d9824 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1815,6 +1815,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 65de45a4447..3c37d1f07ce 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,7 +869,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -909,14 +909,10 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
- if(pgstat_track_vacuum_statistics)
- {
- /* Make extended vacuum stats report for index */
- extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(RelationGetRelid(indrel),
- indrel->rd_rel->relisshared,
- 0, 0, 0, &extVacReport);
- }
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared,
+ &extVacReport);
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 9c2443e1ecd..183f7514d2d 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 6cb9077a27f..0fc16a58210 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -479,6 +479,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 65207d30378..80e6c7c229a 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
@@ -485,7 +494,6 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
- memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bf7ab345be0..817372f9cec 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
@@ -205,50 +203,17 @@ pgstat_drop_relation(Relation rel)
}
}
-/* ---------
- * pgstat_report_vacuum_error() -
- *
- * Tell the collector about an (auto)vacuum interruption.
- * ---------
- */
-void
-pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
-{
- PgStat_EntryRef *entry_ref;
- PgStatShared_Relation *shtabentry;
- PgStat_StatTabEntry *tabentry;
- Oid dboid = MyDatabaseId;
- PgStat_StatDBEntry *dbentry; /* pending database entry */
-
- if (!pgstat_track_counts)
- return;
-
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
- dboid, tableoid, false);
-
- shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
- tabentry = &shtabentry->stats;
-
- tabentry->vacuum_ext.type = m_type;
- pgstat_unlock_entry(entry_ref);
-
- dbentry = pgstat_prep_database_pending(dboid);
- dbentry->vacuum_ext.errors++;
- dbentry->vacuum_ext.type = m_type;
-}
-
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params)
+ TimestampTz starttime)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
- PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -270,8 +235,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
- pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
-
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -307,16 +270,6 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
-
- if (dboid != InvalidOid)
- {
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dboid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
-
- pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
- pgstat_unlock_entry(entry_ref);
- }
}
/*
@@ -951,6 +904,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
@@ -1053,60 +1012,4 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_updated = trans->updated_pre_truncdrop;
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
-}
-
-static void
-pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info)
-{
- if(!pgstat_track_vacuum_statistics)
- return;
-
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->total_time += src->total_time;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->errors += src->errors;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_TABLE)
- {
- dst->table.pages_scanned += src->table.pages_scanned;
- dst->table.pages_removed += src->table.pages_removed;
- dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
- dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
- dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->table.tuples_frozen += src->table.tuples_frozen;
- dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
- dst->table.index_vacuum_count += src->table.index_vacuum_count;
- dst->table.missed_dead_pages += src->table.missed_dead_pages;
- dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->tuples_deleted += src->tuples_deleted;
- }
- }
}
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat_vacuum.c b/src/backend/utils/activity/pgstat_vacuum.c
new file mode 100644
index 00000000000..e11f19e46b2
--- /dev/null
+++ b/src/backend/utils/activity/pgstat_vacuum.c
@@ -0,0 +1,215 @@
+#include "postgres.h"
+
+#include "pgstat.h"
+#include "utils/pgstat_internal.h"
+#include "utils/memutils.h"
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+bool pgstat_track_vacuum_statistics_for_relations = false;
+
+#define ACCUMULATE_FIELD(field) dst->field += src->field;
+
+#define ACCUMULATE_SUBFIELD(substruct, field) \
+ (dst->substruct.field += src->substruct.field)
+
+static void
+pgstat_accumulate_common(PgStat_CommonCounts *dst, const PgStat_CommonCounts *src)
+{
+ ACCUMULATE_FIELD(total_blks_read);
+ ACCUMULATE_FIELD(total_blks_hit);
+ ACCUMULATE_FIELD(total_blks_dirtied);
+ ACCUMULATE_FIELD(total_blks_written);
+
+ ACCUMULATE_FIELD(blks_fetched);
+ ACCUMULATE_FIELD(blks_hit);
+
+ ACCUMULATE_FIELD(wal_records);
+ ACCUMULATE_FIELD(wal_fpi);
+ ACCUMULATE_FIELD(wal_bytes);
+
+ ACCUMULATE_FIELD(blk_read_time);
+ ACCUMULATE_FIELD(blk_write_time);
+ ACCUMULATE_FIELD(delay_time);
+ ACCUMULATE_FIELD(total_time);
+
+ ACCUMULATE_FIELD(tuples_deleted);
+ ACCUMULATE_FIELD(wraparound_failsafe_count);
+}
+
+static void
+pgstat_accumulate_extvac_stats_relations(PgStat_VacuumRelationCounts *dst, PgStat_VacuumRelationCounts *src)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB && src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ ACCUMULATE_SUBFIELD(common, blks_fetched);
+ ACCUMULATE_SUBFIELD(common, blks_hit);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(table, pages_scanned);
+ ACCUMULATE_SUBFIELD(table, pages_removed);
+ ACCUMULATE_SUBFIELD(table, vm_new_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, tuples_frozen);
+ ACCUMULATE_SUBFIELD(table, recently_dead_tuples);
+ ACCUMULATE_SUBFIELD(table, index_vacuum_count);
+ ACCUMULATE_SUBFIELD(table, missed_dead_pages);
+ ACCUMULATE_SUBFIELD(table, missed_dead_tuples);
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(index, pages_deleted);
+ }
+}
+
+static void
+pgstat_accumulate_extvac_stats_db(PgStat_VacuumDBCounts *dst, PgStat_VacuumDBCounts *src)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+ dst->errors += src->errors;
+}
+
+/*
+ * Report that the table was just vacuumed and flush statistics.
+ */
+void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_VacuumRelation *shtabentry;
+ PgStatShared_VacuumDB *shdbentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_RELATION,
+ dboid, tableoid, false);
+ shtabentry = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+ pgstat_accumulate_extvac_stats_relations(&shtabentry->stats, params);
+
+ pgstat_unlock_entry(entry_ref);
+
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_DB,
+ dboid, InvalidOid, false);
+
+ shdbentry = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ pgstat_accumulate_common(&shdbentry->stats.common, ¶ms->common);
+
+ pgstat_unlock_entry(entry_ref);
+}
+
+/*
+ * Flush out pending stats for the entry
+ *
+ * If nowait is true, this function returns false if lock could not
+ * immediately acquired, otherwise true is returned.
+ */
+bool
+pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumRelation *shtabstats;
+ PgStat_RelationVacuumPending *pendingent; /* table entry of shared stats */
+
+ pendingent = (PgStat_RelationVacuumPending *) entry_ref->pending;
+ shtabstats = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts.
+ */
+ if (pg_memory_is_all_zeros(&pendingent,
+ sizeof(struct PgStat_RelationVacuumPending)))
+ return true;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ {
+ return false;
+ }
+
+ pgstat_accumulate_extvac_stats_relations(&(shtabstats->stats), &(pendingent->counts));
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the vacuum collected statistics for one relation or NULL.
+ */
+PgStat_VacuumRelationCounts *
+pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid)
+{
+ return (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_RELATION, dbid, relid);
+}
+
+PgStat_VacuumDBCounts *
+pgstat_fetch_stat_vacuum_dbentry(Oid dbid)
+{
+ return (PgStat_VacuumDBCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_DB, dbid, InvalidOid);
+}
+
+bool
+pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumDB *sharedent;
+ PgStat_VacuumDBCounts *pendingent;
+
+ pendingent = (PgStat_VacuumDBCounts *) entry_ref->pending;
+ sharedent = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ /* The entry was successfully flushed, add the same to database stats */
+ pgstat_accumulate_extvac_stats_db(&(sharedent->stats), pendingent);
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Find or create a local PgStat_VacuumDBCounts entry for dboid.
+ */
+PgStat_VacuumDBCounts *
+pgstat_prep_vacuum_database_pending(Oid dboid)
+{
+ PgStat_EntryRef *entry_ref;
+
+ /*
+ * This should not report stats on database objects before having
+ * connected to a database.
+ */
+ Assert(!OidIsValid(dboid) || OidIsValid(MyDatabaseId));
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_VACUUM_DB, dboid, InvalidOid,
+ NULL);
+
+ if(entry_ref == NULL)
+ return NULL;
+
+ return entry_ref->pending;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1c39ada2c3e..68a99d5db97 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2267,7 +2267,6 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
-
/*
* Get the vacuum statistics for the heap tables.
*/
@@ -2277,102 +2276,42 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
-
- Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- tabentry = pgstat_fetch_stat_tabentry(relid);
-
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = pending;
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
@@ -2380,28 +2319,28 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
@@ -2418,100 +2357,60 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
+ PgStat_VacuumRelationCounts allzero;
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
-
- Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
-
- tabentry = pgstat_fetch_stat_tabentry(relid);
-
- if (tabentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(tabentry->vacuum_ext);
- }
+ extvacuum = pending;
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
@@ -2525,90 +2424,52 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
Oid dbid = PG_GETARG_OID(0);
- PgStat_StatDBEntry *dbentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumDBCounts *extvacuum;
+ PgStat_VacuumDBCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
char buf[256];
int i = 0;
- ExtVacReport allzero;
-
- /* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
-
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
- NUMERICOID, -1, 0);
+ PgStat_VacuumDBCounts allzero;
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
- FLOAT8OID, -1, 0);
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
- FLOAT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
- INT4OID, -1, 0);
+ pending = pgstat_fetch_stat_vacuum_dbentry(dbid);
- Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
-
- BlessTupleDesc(tupdesc);
-
- dbentry = pgstat_fetch_stat_dbentry(dbid);
-
- if (dbentry == NULL)
+ if (pending == NULL)
{
/* If the subscription is not found, initialise its stats */
- memset(&allzero, 0, sizeof(ExtVacReport));
+ memset(&allzero, 0, sizeof(PgStat_VacuumDBCounts));
extvacuum = &allzero;
}
else
- {
- extvacuum = &(dbentry->vacuum_ext);
- }
-
- i = 0;
+ extvacuum = pending;
values[i++] = ObjectIdGetDatum(dbid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
values[i++] = Int32GetDatum(extvacuum->errors);
Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index dcc542750b8..bc9df1433c2 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -432,5 +432,5 @@ extern double anl_get_next_S(double t, int n, double *stateptr);
extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx *counters);
extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx *counters, ExtVacReport *report);
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 6952f833d45..475ce581674 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -116,46 +116,100 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_TABLE = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
+ * PgStat_TableCounts The actual per-table counts kept by a backend
*
- * ExtVacReport
+ * This struct should contain only actual event counters, because we make use
+ * of pg_memory_is_all_zeros() to detect whether there are any stats updates
+ * to apply.
*
- * Additional statistics of vacuum processing over a relation.
- * pages_removed is the amount by which the physically shrank,
- * if any (ie the change in its total size on disk)
- * pages_deleted refer to free space within the index file
+ * It is a component of PgStat_TableStatus (within-backend state).
+ *
+ * Note: for a table, tuples_returned is the number of tuples successfully
+ * fetched by heap_getnext, while tuples_fetched is the number of tuples
+ * successfully fetched by heap_fetch under the control of bitmap indexscans.
+ * For an index, tuples_returned is the number of index entries returned by
+ * the index AM, while tuples_fetched is the number of tuples successfully
+ * fetched by heap_fetch under the control of simple indexscans for this index.
+ *
+ * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
+ * actions, regardless of whether the transaction committed. delta_live_tuples,
+ * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
+ * Note that delta_live_tuples and delta_dead_tuples can be negative!
* ----------
*/
-typedef struct ExtVacReport
+typedef struct PgStat_TableCounts
{
- /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
- int64 total_blks_read;
- int64 total_blks_hit;
- int64 total_blks_dirtied;
- int64 total_blks_written;
+ PgStat_Counter numscans;
- /* blocks missed and hit for just the heap during a vacuum of specific relation */
- int64 blks_fetched;
- int64 blks_hit;
+ PgStat_Counter tuples_returned;
+ PgStat_Counter tuples_fetched;
- /* Vacuum WAL usage stats */
- int64 wal_records; /* wal usage: number of WAL records */
- int64 wal_fpi; /* wal usage: number of WAL full page images produced */
- uint64 wal_bytes; /* wal usage: size of WAL records produced */
+ PgStat_Counter tuples_inserted;
+ PgStat_Counter tuples_updated;
+ PgStat_Counter tuples_deleted;
+ PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_newpage_updated;
+ bool truncdropped;
+
+ PgStat_Counter delta_live_tuples;
+ PgStat_Counter delta_dead_tuples;
+ PgStat_Counter changed_tuples;
+
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
- /* Time stats. */
- double blk_read_time; /* time spent reading pages, in msec */
- double blk_write_time; /* time spent writing pages, in msec */
- double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
- double total_time; /* total time of a vacuum operation, in msec */
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+} PgStat_TableCounts;
- int64 tuples_deleted; /* tuples deleted by vacuum */
+typedef struct PgStat_CommonCounts
+{
+ /* blocks */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* heap blocks */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* WAL */
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+
+ /* Time */
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+
+ /* tuples */
+ int64 tuples_deleted;
+
+ /* failsafe */
+ int32 wraparound_failsafe_count;
+} PgStat_CommonCounts;
- int32 errors;
- int32 wraparound_failsafe_count; /* the number of times to prevent wraparound problem */
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
ExtVacReportType type; /* heap, index, etc. */
@@ -174,16 +228,16 @@ typedef struct ExtVacReport
{
struct
{
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 pages_scanned; /* heap pages examined (not skipped by VM) */
int64 pages_removed; /* heap pages removed by vacuum "truncation" */
int64 pages_frozen; /* pages marked in VM as frozen */
int64 pages_all_visible; /* pages marked in VM as all-visible */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
} table;
@@ -192,61 +246,21 @@ typedef struct ExtVacReport
int64 pages_deleted; /* number of pages deleted by vacuum */
} index;
} /* per_type_stats */;
-} ExtVacReport;
+} PgStat_VacuumRelationCounts;
-/* ----------
- * PgStat_TableCounts The actual per-table counts kept by a backend
- *
- * This struct should contain only actual event counters, because we make use
- * of pg_memory_is_all_zeros() to detect whether there are any stats updates
- * to apply.
- *
- * It is a component of PgStat_TableStatus (within-backend state).
- *
- * Note: for a table, tuples_returned is the number of tuples successfully
- * fetched by heap_getnext, while tuples_fetched is the number of tuples
- * successfully fetched by heap_fetch under the control of bitmap indexscans.
- * For an index, tuples_returned is the number of index entries returned by
- * the index AM, while tuples_fetched is the number of tuples successfully
- * fetched by heap_fetch under the control of simple indexscans for this index.
- *
- * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
- * actions, regardless of whether the transaction committed. delta_live_tuples,
- * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
- * Note that delta_live_tuples and delta_dead_tuples can be negative!
- * ----------
- */
-typedef struct PgStat_TableCounts
+typedef struct PgStat_VacuumRelationStatus
{
- PgStat_Counter numscans;
-
- PgStat_Counter tuples_returned;
- PgStat_Counter tuples_fetched;
-
- PgStat_Counter tuples_inserted;
- PgStat_Counter tuples_updated;
- PgStat_Counter tuples_deleted;
- PgStat_Counter tuples_hot_updated;
- PgStat_Counter tuples_newpage_updated;
- bool truncdropped;
-
- PgStat_Counter delta_live_tuples;
- PgStat_Counter delta_dead_tuples;
- PgStat_Counter changed_tuples;
-
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-
- PgStat_Counter rev_all_visible_pages;
- PgStat_Counter rev_all_frozen_pages;
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
- /*
- * Additional cumulative stat on vacuum operations.
- * Use an expensive structure as an abstraction for different types of
- * relations.
- */
- ExtVacReport vacuum_ext;
-} PgStat_TableCounts;
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ PgStat_CommonCounts common;
+ int32 errors;
+} PgStat_VacuumDBCounts;
/* ----------
* PgStat_TableStatus Per-table status within a backend
@@ -272,6 +286,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -468,8 +488,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
-
- ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -551,8 +569,6 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter rev_all_visible_pages;
PgStat_Counter rev_all_frozen_pages;
-
- ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -760,11 +776,10 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport *params);
+ TimestampTz starttime);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
-extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -895,6 +910,15 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts *params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts *pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts *pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid);
+PgStat_VacuumDBCounts *pgstat_fetch_stat_vacuum_dbentry(Oid dbid);
/*
* Functions in pgstat_wal.c
*/
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index bf75ebcef31..628e65e4a5d 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -447,6 +447,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -615,6 +627,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index eb5f0b3ae6d..52e884fbf8b 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 14f19e5bcbe..ef2d6545953 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2293,77 +2293,81 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
-pg_stat_vacuum_database| SELECT db.oid AS dboid,
- db.datname AS dbname,
- stats.db_blks_read,
- stats.db_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time,
- stats.wraparound_failsafe,
- stats.errors
- FROM pg_database db,
- LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
-pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
- ns.nspname AS schemaname,
- rel.relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_deleted,
- stats.tuples_deleted,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time
- FROM (pg_class rel
- JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
- LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
- WHERE (rel.relkind = 'i'::"char");
-pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
- rel.relname,
- stats.relid,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_scanned,
- stats.pages_removed,
- stats.vm_new_frozen_pages,
- stats.vm_new_visible_pages,
- stats.vm_new_visible_frozen_pages,
- stats.missed_dead_pages,
- stats.tuples_deleted,
- stats.tuples_frozen,
- stats.recently_dead_tuples,
- stats.missed_dead_tuples,
- stats.wraparound_failsafe,
- stats.index_vacuum_count,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time
- FROM (pg_class rel
- JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
- LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
- WHERE (rel.relkind = 'r'::"char");
+pg_stat_vacuum_database| SELECT d.oid AS dboid,
+ d.datname AS dbname,
+ s.db_blks_read,
+ s.db_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time,
+ s.wraparound_failsafe,
+ s.errors
+ FROM pg_database d,
+ LATERAL pg_stat_get_vacuum_database(d.oid) s(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
+pg_stat_vacuum_indexes| SELECT c.oid AS relid,
+ i.oid AS indexrelid,
+ n.nspname AS schemaname,
+ c.relname,
+ i.relname AS indexrelname,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_deleted,
+ s.tuples_deleted,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (((pg_class c
+ JOIN pg_index x ON ((c.oid = x.indrelid)))
+ JOIN pg_class i ON ((i.oid = x.indexrelid)))
+ LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(i.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
+pg_stat_vacuum_tables| SELECT n.nspname AS schemaname,
+ c.relname,
+ s.relid,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_scanned,
+ s.pages_removed,
+ s.vm_new_frozen_pages,
+ s.vm_new_visible_pages,
+ s.vm_new_visible_frozen_pages,
+ s.missed_dead_pages,
+ s.tuples_deleted,
+ s.tuples_frozen,
+ s.recently_dead_tuples,
+ s.missed_dead_tuples,
+ s.wraparound_failsafe,
+ s.index_vacuum_count,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (pg_class c
+ JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(c.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
index 9e5d33342c9..4654a536ad6 100644
--- a/src/test/regress/expected/vacuum_index_statistics.out
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -30,9 +30,9 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- Must be empty.
SELECT *
FROM pg_stat_vacuum_indexes vt
-WHERE vt.relname = 'vestat';
- relid | schemaname | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
--------+------------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+WHERE vt.indexrelname = 'vestat';
+ relid | indexrelid | schemaname | relname | indexrelname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+------------+------------+---------+--------------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
(0 rows)
RESET track_vacuum_statistics;
@@ -55,12 +55,12 @@ ANALYZE vestat;
SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
DELETE FROM vestat WHERE x % 2 = 0;
-- Before the first vacuum execution extended stats view is empty.
-SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+SELECT vt.indexrelname,relpages,pages_deleted,tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | 30 | 0 | 0
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
(1 row)
SELECT relpages AS irp
@@ -72,22 +72,22 @@ CHECKPOINT;
-- The table and index extended vacuum statistics should show us that
-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
-- because of not full table have cleaned up
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | t | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
(1 row)
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- Look into WAL records deltas.
SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey';
diwr | diwb
------+------
t | t
@@ -98,20 +98,20 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
-- it is necessary to check the wal statistics
CHECKPOINT;
-- pages_removed must be increased
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | t | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
(1 row)
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL advance should be detected.
SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
diwr | diwb
@@ -120,7 +120,7 @@ SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
(1 row)
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
DELETE FROM vestat WHERE x % 2 = 0;
-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
@@ -130,7 +130,7 @@ VACUUM FULL vestat;
CHECKPOINT;
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL and other statistics advance should not be detected.
SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
diwr | ifpi | diwb
@@ -138,19 +138,19 @@ SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
t | t | t
(1 row)
-SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | t | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
(1 row)
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
DELETE FROM vestat;
TRUNCATE vestat;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
@@ -158,7 +158,7 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
CHECKPOINT;
-- Store WAL advances into variables after removing all tuples from the table
SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
--There are nothing changed
SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
diwr | ifpi | diwb
@@ -171,12 +171,12 @@ SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
-- in vacuum extended statistics.
-- The pages_frozen, pages_scanned values shouldn't be changed
--
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
- relname | relpages | pages_deleted | tuples_deleted
--------------+----------+---------------+----------------
- vestat_pkey | f | t | t
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+ indexrelname | relpages | pages_deleted | tuples_deleted
+--------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
(1 row)
DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
index 9b7e645187d..57e5420b9b6 100644
--- a/src/test/regress/sql/vacuum_index_statistics.sql
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -27,7 +27,7 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- Must be empty.
SELECT *
FROM pg_stat_vacuum_indexes vt
-WHERE vt.relname = 'vestat';
+WHERE vt.indexrelname = 'vestat';
RESET track_vacuum_statistics;
DROP TABLE vestat CASCADE;
@@ -49,9 +49,9 @@ SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
DELETE FROM vestat WHERE x % 2 = 0;
-- Before the first vacuum execution extended stats view is empty.
-SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+SELECT vt.indexrelname,relpages,pages_deleted,tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
SELECT relpages AS irp
FROM pg_class c
WHERE relname = 'vestat_pkey' \gset
@@ -63,19 +63,19 @@ CHECKPOINT;
-- The table and index extended vacuum statistics should show us that
-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
-- because of not full table have cleaned up
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- Look into WAL records deltas.
SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey';
DELETE FROM vestat;;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
@@ -83,22 +83,22 @@ VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
CHECKPOINT;
-- pages_removed must be increased
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL advance should be detected.
SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
DELETE FROM vestat WHERE x % 2 = 0;
@@ -110,20 +110,20 @@ CHECKPOINT;
-- Store WAL advances into variables
SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
-- WAL and other statistics advance should not be detected.
SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
-SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
-SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
+SELECT vt.indexrelname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid \gset
-- Store WAL advances into variables
-SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
DELETE FROM vestat;
TRUNCATE vestat;
@@ -133,7 +133,7 @@ CHECKPOINT;
-- Store WAL advances into variables after removing all tuples from the table
SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
-FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey' \gset
--There are nothing changed
SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
@@ -143,9 +143,9 @@ SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
-- in vacuum extended statistics.
-- The pages_frozen, pages_scanned values shouldn't be changed
--
-SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+SELECT vt.indexrelname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
FROM pg_stat_vacuum_indexes vt, pg_class c
-WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+WHERE vt.indexrelname = 'vestat_pkey' AND vt.indexrelid = c.oid;
DROP TABLE vestat;
RESET track_vacuum_statistics;
--
2.34.1
[text/x-patch] v25-0005-Add-documentation-about-the-system-views-that-are-us.patch (24.5K, 6-v25-0005-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 4f937350e6f570e9a576abdbc7698b350f4d0288 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 5/5] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 755 +++++++++++++++++++++++++++++++++
1 file changed, 755 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 4187191ea74..183fb0bfce3 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5545,4 +5545,759 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.34.1
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-09-25 15:10 Alena Rybakina <[email protected]>
parent: Bharath Rupireddy <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2025-09-25 15:10 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; Amit Kapila <[email protected]>; Alexander Korotkov <[email protected]>; +Cc: pgsql-hackers; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>
Hi all,
I’ve prepared an extension that adds vacuum statistics [0] (master
branch), and it’s working stably. The attached patch is a core patch
that enables this extension to work.
Right now, I’m experimenting with a core patch. Specifically, in
load_file I can detect whether vacuum_statistics is listed in
shared_preload_libraries and, if so, start collecting vacuum statistics
in the core.
However, I think it would be more reliable to simply add a dedicated
hook for vacuum statistics collection in the core. In my view, an
extension may be loaded but disabled for vacuum statistics collection -
and in that case we shouldn’t gather them.
In general, I’m not entirely happy with the current organization. One
issue that bothers me is having to scan the entire hash table to provide
vacuum statistics for a database, with aggregation.
At the moment, the hash table uses (dboid, reloid, type) as the key.
This could be improved by introducing another hash table keyed by dboid,
with entries containing arrays of the first table’s keys (dboid, reloid,
type) (where dboid is either kept or omitted).
The idea is that we find the relevant array for a given database and
then aggregate its statistics by iterating over the first table using
those keys. I’ve started implementing this approach in the main branch
of the same repository, but
I’m still working out the issues with dynamic memory management.
I also have an idea for effectively splitting statistics into “core” and
“extra.”
Core statistics:
For databases (also collected for tables and indexes): delay_time,
total_time
For tables: pages_scanned, pages_removed, tuples_deleted,
vm_new_frozen_pages, vm_new_visible_pages
For indexes: tuples_deleted, pages_deleted
Extra statistics:
For databases (also collected for tables and indexes): total_blks_read,
total_blks_dirtied, total_blks_written, blks_fetched, blks_hit,
blk_read_time, blk_write_time
For tables: recently_dead_tuples, missed_dead_tuples,
vm_new_visible_frozen_pages, missed_dead_pages, tuples_frozen
WAL statistics (separately for databases and relations):wal_records,
wal_fpi, wal_bytes
I’ve already started drafting the first implementation, but I still need
to carefully handle memory allocation.
Additionally, I’m considering letting users define which databases,
schemas, or tables/relations should have vacuum statistics collected. I
believe this could be valuable for large, high-load systems.
For example, the core statistics might show that a particular database
is frequently vacuumed - so we could then focus on tracking only that
one. Similarly, if certain tables are heavily updated by backends and
vacuumed often, we could target those specifically. Conceptually, this
would act like a filter, but at this point, it’s just an idea for a
future improvement.
This is the direction I’m planning to take with the patch. If you have
alternative ideas about how to organize the code, I’d be glad to hear them!
On 25.09.2025 03:03, Bharath Rupireddy wrote:
> Hi,
>
> On Mon, May 12, 2025 at 5:30 AM Amit Kapila <[email protected]> wrote:
>> On Fri, May 9, 2025 at 5:34 PM Alena Rybakina <[email protected]> wrote:
>>> I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and
>>> this allows us to solve the problem with the memory consumed.
>>>
>> I think this patch is trying to collect data similar to what we do for
>> pg_stat_statements for SQL statements. So, can't we follow a similar
>> idea such that these additional statistics will be collected once some
>> external module like pg_stat_statements is enabled? That module should
>> be responsible for accumulating and resetting the data, so we won't
>> have this memory consumption issue.
>>
>> BTW, how will these new statistics be used to autotune a vacuum? And
>> do we need all the statistics proposed by this patch?
> Thanks for working on this. I agree with the general idea of having
> minimal changes to the core. I think a simple approach would be to
> have a hook in heap_vacuum_rel at the end, where vacuum stats are
> prepared in a buffer for emitting LOG messages. External modules can
> then handle storing, rotating, interpreting, aggregating (per
> relation/per database), and exposing the stats to end-users via SQL.
> The core can define a common data structure, fill it, and send it to
> external modules. I haven't had a chance to read the whole thread or
> review the patches; I'm sure this has been discussed.
>
As for how this may help databases in practice, I think it deserves a
separate thread once the vacuum statistics patch is pushed.
In short, such statistics are essential to understand the real impact of
vacuum on system load.
For example:
If vacuum runs very frequently on a table or index, this might point to
table or index bloat, or to overly aggressive configuration.
Conversely, if vacuum freezes or removes very few tuples, it may suggest
that vacuum is not aggressive enough, or that delays are set too high.
If missed_dead_pages and missed_dead_tuples are high compared to
tuples_deleted, that may indicate vacuum can’t obtain a INDEX CLEANUP
LOCK or doesn’t retry due to long delays.
Statistics related to wraparound activity can also hint that autovacuum
settings require adjustment.
It’s also possible that this system could be made more automatic in the
future, but I haven’t fully worked out how yet. I think that discussion
belongs in a separate thread once vacuum statistics themselves are
committed.
[0] https://github.com/Alena0704/vacuum_statistics/tree/master
-------------
Best regards,
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] vacuum_statistics.patch (38.1K, 2-vacuum_statistics.patch)
download | inline diff:
From 81eff11a045d15a45f41805aea5ac36438acafa6 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 4 Sep 2025 18:16:52 +0300
Subject: Core patch.
---
src/backend/access/heap/vacuumlazy.c | 308 ++++++++++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 12 +
src/backend/utils/activity/pgstat_relation.c | 36 ++-
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/backend/utils/error/elog.c | 13 +
src/include/catalog/pg_proc.dat | 8 +
src/include/commands/vacuum.h | 26 ++
src/include/pgstat.h | 118 ++++++-
src/include/utils/elog.h | 1 +
src/test/regress/expected/rules.out | 12 +-
13 files changed, 546 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 932701d8420..a56cb0222fa 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -289,6 +289,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -407,6 +409,10 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to prevent anti-wraparound shutdown */
+
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -474,6 +480,208 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ PgStat_VacuumRelationCounts *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->common.total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->common.total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->common.total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->common.total_blks_written += bufusage.shared_blks_written;
+
+ report->common.wal_records += walusage.wal_records;
+ report->common.wal_fpi += walusage.wal_fpi;
+ report->common.wal_bytes += walusage.wal_bytes;
+
+ report->common.blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->common.blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->common.blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->common.blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->common.delay_time += VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->common.total_time += secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->common.blks_fetched +=
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->common.blks_hit +=
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
+
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ /* Set initial values for common heap and index statistics*/
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacStats)
+{
+ /* Fill heap-specific extended stats fields */
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacStats->table.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+
+}
+
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts *extVacIdxStats)
+{
+
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
/*
@@ -632,6 +840,13 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
char **indnames = NULL;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts ExtVacReport;
+ PgStat_VacuumRelationCounts allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(PgStat_VacuumRelationCounts));
+ ExtVacReport = allzero;
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -652,6 +867,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -668,6 +884,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -676,6 +893,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(PgStat_VacuumRelationCounts));
+
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
vac_open_indexes(vacrel->rel, RowExclusiveLock, &vacrel->nindexes,
@@ -776,6 +995,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->aggressive = vacuum_get_cutoffs(rel, params, &vacrel->cutoffs);
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ vacrel->wraparound_failsafe_count = 0;
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
@@ -924,6 +1144,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &ExtVacReport);
+
/*
* Report results to the cumulative stats system, too.
*
@@ -934,12 +1157,20 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
+ /* Make generic extended vacuum stats report and
+ * fill heap-specific extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &ExtVacReport);
+ accumulate_heap_vacuum_statistics(vacrel, &ExtVacReport);
+
pgstat_report_vacuum(RelationGetRelid(rel),
- rel->rd_rel->relisshared,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ rel->rd_rel->relisshared,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &ExtVacReport);
+
pgstat_progress_end_command();
if (instrument)
@@ -2631,10 +2862,20 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ PgStat_VacuumRelationCounts PgStat_VacuumRelationCounts;
+
+ memset(&PgStat_VacuumRelationCounts, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
+ extvac_stats_end(vacrel->rel, &counters, &PgStat_VacuumRelationCounts);
+ accumulate_idxs_vacuum_statistics(vacrel, &PgStat_VacuumRelationCounts);
+
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
* that parallel VACUUM only gets the precheck and this postcheck.
@@ -2961,6 +3202,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[2] = {0, 0};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count ++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3043,10 +3285,20 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ PgStat_VacuumRelationCounts PgStat_VacuumRelationCounts;
+
+ memset(&PgStat_VacuumRelationCounts, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
estimated_count);
+
+ extvac_stats_end(vacrel->rel, &counters, &PgStat_VacuumRelationCounts);
+ accumulate_idxs_vacuum_statistics(vacrel, &PgStat_VacuumRelationCounts);
}
/* Reset the progress counters */
@@ -3072,6 +3324,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts PgStat_VacuumRelationCounts;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3090,6 +3347,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3098,6 +3356,16 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &PgStat_VacuumRelationCounts);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &PgStat_VacuumRelationCounts);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &PgStat_VacuumRelationCounts);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3122,6 +3390,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts PgStat_VacuumRelationCounts;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3141,12 +3414,22 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &PgStat_VacuumRelationCounts);
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &PgStat_VacuumRelationCounts);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &PgStat_VacuumRelationCounts);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3759,6 +4042,9 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, errinfo->rel->rd_rel->relisshared, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3774,6 +4060,9 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, errinfo->rel->rd_rel->relisshared, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3789,16 +4078,25 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, errinfo->rel->rd_rel->relisshared, PGSTAT_EXTVAC_INDEX);
+
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, errinfo->rel->rd_rel->relisshared, PGSTAT_EXTVAC_INDEX);
+
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, errinfo->rel->rd_rel->relisshared, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 953ad4a4843..a21e77cd551 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index c77fa0234bb..b3242dc6f5a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -716,7 +716,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_vacuum_time(C.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(C.oid) as rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) as rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 733ef40ae7c..d8776ff1901 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -116,6 +116,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2533,6 +2536,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 0feea1d30ec..b5461ec661b 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,12 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, 0, &extVacReport);
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1054,6 +1065,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 69df741cbf6..33a4009f746 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -203,13 +203,39 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, bool shared, ExtVacReportType m_type)
+{
+ PgStat_VacuumRelationCounts params;
+
+ if (!pgstat_track_counts)
+ return;
+
+ if (set_report_vacuum_hook)
+ {
+ memset(¶ms, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ params.common.interrupts_count++;
+
+ (*set_report_vacuum_hook) (tableoid, shared, ¶ms);
+ }
+}
+
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime)
+ TimestampTz starttime, PgStat_VacuumRelationCounts *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -235,6 +261,11 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ if (set_report_vacuum_hook)
+ {
+ (*set_report_vacuum_hook) (tableoid, shared, params);
+ }
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -881,6 +912,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..9482bf80721 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index b7b9692f8c8..f0ecf86e514 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1627,6 +1627,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 118d6da1ace..e0c7cf29b3a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12576,4 +12576,12 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 14eeccbd718..bc9df1433c2 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -295,6 +296,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -327,6 +348,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
@@ -407,4 +429,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, PgStat_VacuumRelationCounts *report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f402b17295c..15da1f27654 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -100,6 +100,97 @@ typedef struct PgStat_FunctionCallUsage
instr_time start;
} PgStat_FunctionCallUsage;
+
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ /* blocks */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* heap blocks */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* WAL */
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+
+ /* Time */
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+
+ /* failsafe */
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+} PgStat_CommonCounts;
+
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted;
+ } table;
+ struct
+ {
+ int64 tuples_deleted;
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
+} PgStat_VacuumRelationCounts;
+
+
/* ----------
* PgStat_BackendSubEntry Non-flushed subscription stats.
* ----------
@@ -153,6 +244,9 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -211,7 +305,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB7
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB8
typedef struct PgStat_ArchiverStats
{
@@ -453,6 +547,9 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autovacuum_time;
PgStat_Counter total_analyze_time;
PgStat_Counter total_autoanalyze_time;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_StatTabEntry;
/* ------
@@ -660,10 +757,11 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
- TimestampTz starttime);
+ TimestampTz starttime, PgStat_VacuumRelationCounts *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
+extern void pgstat_report_vacuum_error(Oid tableoid, bool shared, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -711,6 +809,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -838,4 +947,9 @@ extern PGDLLIMPORT PgStat_Counter pgStatTransactionIdleTime;
/* updated by the traffic cop and in errfinish() */
extern PGDLLIMPORT SessionEndType pgStatSessionEndCause;
+/* Hook for plugins to get control in set_rel_pathlist() */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared, PgStat_VacuumRelationCounts *params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
+
+
#endif /* PGSTAT_H */
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 675f4f5f469..356dadd6b0a 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrcode(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..4731ca2121e 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1833,7 +1833,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_vacuum_time(c.oid) AS total_vacuum_time,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
- pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time
+ pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2232,7 +2234,9 @@ pg_stat_sys_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2284,7 +2288,9 @@ pg_stat_user_tables| SELECT relid,
total_vacuum_time,
total_autovacuum_time,
total_analyze_time,
- total_autoanalyze_time
+ total_autoanalyze_time,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.34.1
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2025-12-20 23:36 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2025-12-20 23:36 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; [email protected]; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>; Alena Rybakina <[email protected]>
Hi,
I’ve added some changes to one of the approaches and also did additional
cleanup and stabilization work on the vacuum statistics tests. Specifically:
* I moved the vacuum statistics tests into the tests tab and made them
more stable. For slower machines, vacuum is now triggered inside the
statistics wait function. Previously, some backends didn’t have
enough time to release the lock, which could lead to differences
because the vacuum hadn’t fully completed yet.
* I also ran the backend tests and fixed a couple of minor issues
along the way.
* I ran pgindent to clean up and normalize the formatting.
For now, I’ve temporarily removed collecting statistics related to
database-level errors when vacuum is forced to stop. I’m currently stuck
on how to properly expose statistics for cluster-level objects, since
their dbid is 0.
At the moment, only the second test still looks odd, and I haven’t fully
figured out why yet. It seems like aggressive vacuum can no longer be
triggered the same way as before with the current gucs, but I’m still
investigating this.
Best regards,
Alena Rybakina
From f96d5079774fe129fff32761bba4ab9089e491bd Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 9 Dec 2025 09:56:34 +0300
Subject: [PATCH 1/5] Machinery for grabbing an extended vacuum statistics on
table relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen is a number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible is a number of pages that are marked as all-visible in vm during
vacuum.
wraparound_failsafe_count is a number of times when the vacuum starts urgent cleanup
to prevent wraparound problem which is critical for the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 145 ++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 52 +-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 46 +-
src/backend/utils/adt/pgstatfuncs.c | 86 +++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 18 +
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 92 ++-
.../vacuum-extending-in-repetable-read.out | 53 ++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 53 ++
.../t/050_vacuum_extending_basic_test.pl | 571 ++++++++++++++++++
.../t/051_vacuum_extending_freeze_test.pl | 395 ++++++++++++
src/test/regress/expected/rules.out | 44 +-
src/test/regress/parallel_schedule | 2 +-
18 files changed, 1565 insertions(+), 10 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/recovery/t/050_vacuum_extending_basic_test.pl
create mode 100644 src/test/recovery/t/051_vacuum_extending_freeze_test.pl
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 30778a15639..66e09d0a0cf 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -289,6 +289,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -407,6 +408,10 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to
+ * prevent anti-wraparound
+ * shutdown */
} LVRelState;
@@ -418,6 +423,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -487,6 +504,102 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ TimestampTz starttime;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ ExtVacReport * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
@@ -645,6 +758,13 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
char **indnames = NULL;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -673,6 +793,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
pgstat_progress_update_param(PROGRESS_VACUUM_STARTED_BY,
PROGRESS_VACUUM_STARTED_BY_MANUAL);
+ extvac_stats_start(rel, &extVacCounters);
+
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -689,6 +811,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -797,6 +920,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->aggressive = vacuum_get_cutoffs(rel, params, &vacrel->cutoffs);
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ vacrel->wraparound_failsafe_count = 0;
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
@@ -951,6 +1075,23 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -965,7 +1106,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime);
+ starttime,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -3019,6 +3161,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index d14588e92ae..3030242d98e 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -161,6 +162,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 0a0f95f6bb9..ffb407d414f 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -727,7 +727,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1452,3 +1454,51 @@ REVOKE ALL ON pg_aios FROM PUBLIC;
GRANT SELECT ON pg_aios TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_aios() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.vm_new_frozen_pages AS vm_new_frozen_pages,
+ stats.vm_new_visible_pages AS vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ stats.missed_dead_pages AS missed_dead_pages,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.recently_dead_tuples AS recently_dead_tuples,
+ stats.missed_dead_tuples AS missed_dead_tuples,
+
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 0528d1b6ecb..dd519447387 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -117,6 +117,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2536,6 +2539,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 8a37c08871a..114cd7c31d3 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1054,6 +1054,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 55a10c299db..361713479e8 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
+ bool accumulate_reltype_specific_info);
/*
@@ -208,7 +210,7 @@ pgstat_drop_relation(Relation rel)
*/
void
pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
- PgStat_Counter deadtuples, TimestampTz starttime)
+ PgStat_Counter deadtuples, TimestampTz starttime, ExtVacReport * params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -234,6 +236,8 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -880,6 +884,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -1009,3 +1016,40 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
+ dst->vm_new_visible_pages += src->vm_new_visible_pages;
+ dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->missed_dead_pages += src->missed_dead_pages;
+ dst->missed_dead_tuples += src->missed_dead_tuples;
+
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index ef6fffe60b9..d7dfda0c1a7 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2307,3 +2313,83 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index dc9e2255f8a..867638fe74b 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -669,6 +669,7 @@
#track_wal_io_timing = off
#track_functions = none # none, pl, all
#stats_fetch_consistency = cache # cache, none, snapshot
+#track_vacuum_statistics = off
# - Monitoring -
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fd9448ec7b9..915a5a7822f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12612,4 +12612,22 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int4,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 1f3290c7fbf..6b997bc7fb1 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -332,6 +332,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 6714363144a..46d12fa3bd0 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -114,6 +114,66 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /*
+ * number of blocks missed, hit, dirtied and written during a vacuum of
+ * specific relation
+ */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /*
+ * blocks missed and hit for just the heap during a vacuum of specific
+ * relation
+ */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images
+ * produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay
+ * point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as
+ * all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to
+ * failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still
+ * visible to some transaction */
+ int64 index_vacuum_count; /* the number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to
+ * prevent anti-wraparound
+ * shutdown */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -156,6 +216,15 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations. Use an expensive
+ * structure as an abstraction for different types of relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -214,7 +283,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -378,6 +447,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -461,8 +532,12 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autovacuum_time;
PgStat_Counter total_analyze_time;
PgStat_Counter total_autoanalyze_time;
-
TimestampTz stat_reset_time;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -671,7 +746,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
- TimestampTz starttime);
+ TimestampTz starttime, ExtVacReport * params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -722,6 +797,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..87f7e40b4a6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index f2e067b1fbc..1c231418706 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -98,6 +98,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5893d89573d
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,53 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/recovery/t/050_vacuum_extending_basic_test.pl b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..7e25a3fe63f
--- /dev/null
+++ b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
@@ -0,0 +1,571 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across tables using:
+#
+# • pg_stat_vacuum_tables
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server logging level for the test
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls pg_stat_vacuum_tables until the table-level counters exceed
+# the provided baselines, or until the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = $args{tab_tuples_deleted} or 0;
+ my $tab_wal_records = $args{tab_wal_records} or 0;
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $pages_frozen = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $pages_frozen_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table,
+# storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($pages_frozen, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $pages_frozen_prev = $pages_frozen;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " pages_frozen = $pages_frozen_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " pages_frozen = $pages_frozen\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats during mismatch
+#
+# Print current values and old values of relevant vacuum counters for the test
+# table, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_error_base_tab_vacuum_statistics {
+
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_frozen_pages, tuples_deleted, pages_scanned, pages_removed
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($cur_pages_frozen, $cur_tuples_deleted, $cur_pages_scanned, $cur_pages_removed) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR TABLE:\n" .
+ " Baseline:\n" .
+ " pages_frozen = $pages_frozen\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " Current:\n" .
+ " pages_frozen = $cur_pages_frozen\n" .
+ " tuples_deleted = $cur_tuples_deleted\n" .
+ " pages_scanned = $cur_pages_scanned\n" .
+ " pages_removed = $cur_pages_removed\n"
+ );
+}
+
+sub fetch_error_wal_tab_vacuum_statistics {
+
+ my $wal_raw = $node->safe_psql(
+ $dbname,
+ "SELECT wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $wal_raw =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($cur_wal_rec, $cur_wal_bytes, $cur_wal_fpi) = split /\s+/, $wal_raw;
+
+ diag(
+ "WAL STATS MISMATCH FOR TABLE:\n" .
+ " Baseline:\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " Current:\n" .
+ " wal_records = $cur_wal_rec\n" .
+ " wal_bytes = $cur_wal_bytes\n" .
+ " wal_fpi = $cur_wal_fpi\n"
+ );
+}
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM, and wait for stats to advance' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds after vacuuming half-deleted table";
+
+#------------------------------------------------------------------------------
+# Check statistics after half-table delete
+#------------------------------------------------------------------------------
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM, and wait for stats to advance' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds after vacuuming all-deleted table";
+
+#------------------------------------------------------------------------------
+# Check statistics after full delete
+#------------------------------------------------------------------------------
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted,
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Verify statistics after updating tuples and vacuuming
+#------------------------------------------------------------------------------
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => $wal_records_prev
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Verify statistics after updating full table, vacuum and trancation
+#------------------------------------------------------------------------------
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => $wal_records
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Verify statistics after table vacuum and trancation
+#------------------------------------------------------------------------------
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#-------------------------------------------------------------------------------------------------------
+# Test 8: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap elation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) = 1 FROM pg_stat_get_vacuum_tables($reloid);"
+);
+ok($base_stats eq 't', 'heap vacuum stats return from the current relation and database as expected');
+
+#------------------------------------------------------------------------------
+# Test 9: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 0
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+ok($base_stats eq 't', 'check the printing heap vacuum extended statistics from another database are not available');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+# Check if we can get vacuum statistics for cluster relations (dbid = 0)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) = 1
+ FROM pg_stat_get_vacuum_tables($reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+#------------------------------------------------------------------------------
+# Test 11: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*) = 0
+ FROM pg_stat_vacuum_tables WHERE relid = 0;
+ }
+);
+ok($base_stats eq 't', 'pg_stat_vacuum_tables correctly returns no rows for OID = 0');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;"
+);
+
+$node->stop;
+
+done_testing();
diff --git a/src/test/recovery/t/051_vacuum_extending_freeze_test.pl b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..a9b5d6cb739
--- /dev/null
+++ b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
@@ -0,0 +1,395 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats system using TAP
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server for aggressive freezing behavior used by the test
+# These settings ensure that VACUUM always freezes pages aggressively:
+# - vacuum_freeze_min_age = 0: freeze tuples as soon as possible (no age requirement)
+# - vacuum_freeze_table_age = 0: always perform aggressive scan (scan all pages)
+# - vacuum_multixact_freeze_min_age = 0: freeze multixacts as soon as possible
+# - vacuum_multixact_freeze_table_age = 0: always perform aggressive scan for multixacts
+# - vacuum_max_eager_freeze_failure_rate = 1.0: enable aggressive eager scanning (100% of pages)
+# - vacuum_failsafe_age = 0: disable failsafe (for testing)
+# - vacuum_multixact_failsafe_age = 0: disable multixact failsafe (for testing)
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Enable necessary settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+# wait_for_vacuum_stats
+#
+# Polls pg_stat_vacuum_tables until the named columns exceed the provided
+# baseline values or until timeout. Callers should pass:
+#
+# tab_frozen_column => 'vm_new_frozen_pages' # column name (string) or 'rev_all_frozen_pages'
+# tab_visible_column => 'vm_new_visible_pages' # column name (string) or 'rev_all_visible_pages'
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 or 1 # if true, run vacuum_sql before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_frozen_column = $args{tab_frozen_column};
+ my $tab_visible_column = $args{tab_visible_column};
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count};
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count};
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ $sql = "
+ SELECT ($tab_frozen_column > $tab_all_frozen_pages_count AND
+ $tab_visible_column > $tab_all_visible_pages_count)
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_rev_all_frozen_pages(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_rev_all_visible_pages(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ # sub-second sleep
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_frozen_pages;
+my $vm_new_visible_pages;
+
+my $rev_all_frozen_pages;
+my $rev_all_visible_pages;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ $vm_new_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_frozen_pages
+ FROM pg_stat_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $vm_new_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_pages
+ FROM pg_stat_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats during mismatch
+#
+# Print current values and old values of relevant vacuum counters for the test
+# table, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_error_tab_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "tab_column required"
+ unless exists $args{tab_column} && defined $args{tab_column};
+ die "tab_value required"
+ unless exists $args{tab_value};
+
+ my $tab_column = $args{tab_column};
+ my $tab_value = $args{tab_value};
+
+ # fetch actual base vacuum statistics
+ my $cur_value = $node->safe_psql(
+ $dbname,
+ "SELECT $tab_column
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ diag("MISMATCH FOR $tab_column: the current value is $cur_value, while it should be $tab_value");
+}
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+$node->safe_psql($dbname, q{
+ SELECT pg_stat_force_next_flush();
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+# Poll the stats view until the expected deltas appear or timeout.
+# We do not expect rev_all_* counters to change here, so we pass -1 for them.
+$updated = wait_for_vacuum_stats(
+ tab_frozen_column => 'vm_new_frozen_pages',
+ tab_visible_column => 'vm_new_visible_pages',
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_frozen_pages and vm_new_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Verify initial statistics after vacuum
+#------------------------------------------------------------------------------
+
+$res = $node->safe_psql($dbname, q{
+ SELECT vm_new_frozen_pages > 0 FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+});
+ok($res eq 't', 'vacuum froze some pages, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_frozen_pages', tab_value => $vm_new_frozen_pages,);
+
+$res = $node->safe_psql($dbname, q{
+ SELECT vm_new_visible_pages > 0 FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+});
+ok($res eq 't', 'vacuum marked pages all-visible, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_visible_pages', tab_value =>$vm_new_visible_pages,);
+
+$res = $node->safe_psql($dbname, q{
+ SELECT pg_stat_get_rev_all_frozen_pages(c.oid) = 0
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';
+});
+ok($res eq 't', 'vacuum did not increase frozen-page revision count, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_frozen_pages', tab_value => 0,);
+
+$res = $node->safe_psql($dbname, q{
+ SELECT pg_stat_get_rev_all_visible_pages(c.oid) = 0
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';
+});
+ok($res eq 't', 'vacuum did not increase visible-page revision count, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_visible_pages', tab_value => 0,);
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+# Poll until stats update or timeout.
+# We do not expect vm_new_frozen_pages or vm_new_visible_pages to change here,
+# so we pass -1 for those counters.
+$updated = wait_for_vacuum_stats(
+ tab_frozen_column => 'rev_all_frozen_pages',
+ tab_visible_column => 'rev_all_visible_pages',
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Check updated statistics after backend activity
+#------------------------------------------------------------------------------
+
+$res = $node->safe_psql($dbname,
+ "SELECT vm_new_frozen_pages = $vm_new_frozen_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+);
+ok($res eq 't', 'backend activity did not increase the frozen-page count') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_frozen_pages', tab_value => $vm_new_frozen_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT vm_new_visible_pages = $vm_new_visible_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+);
+ok($res eq 't', 'backend activity did not increase the all-visible page count') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_visible_pages', tab_value => $vm_new_visible_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid) > $rev_all_frozen_pages
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';"
+);
+ok($res eq 't', 'backend activity increased frozen-page revision count') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_frozen_pages', tab_value => $rev_all_frozen_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid) > $rev_all_visible_pages
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';"
+);
+ok($res eq 't', 'backend activity increased visible-page revision count') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_visible_pages', tab_value => $rev_all_visible_pages,);
+
+#------------------------------------------------------------------------------
+# Update saved snapshots
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+
+$node->safe_psql($dbname, q{ VACUUM (FREEZE, VERBOSE) vestat; });
+
+# Poll until stats update or timeout.
+# We pass current snapshot values for vm_new_frozen_pages/vm_new_visible_pages and expect rev counters unchanged.
+$updated = wait_for_vacuum_stats(
+ tab_frozen_column => 'vm_new_frozen_pages',
+ tab_visible_column => 'vm_new_visible_pages',
+ tab_all_frozen_pages_count => $vm_new_frozen_pages,
+ tab_all_visible_pages_count => $vm_new_visible_pages,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_frozen_pages and vm_new_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Verify statistics after final vacuum
+# Check updated stats after backend work
+#------------------------------------------------------------------------------
+$res = $node->safe_psql($dbname,
+ "SELECT vm_new_frozen_pages > $vm_new_frozen_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+);
+ok($res eq 't', 'vacuum froze some pages after backend activity, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_frozen_pages', tab_value => $vm_new_frozen_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT vm_new_visible_pages > $vm_new_visible_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+);
+ok($res eq 't', 'vacuum marked pages all-visible after backend activity, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_visible_pages', tab_value => $vm_new_visible_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid) = $rev_all_frozen_pages
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';"
+);
+ok($res eq 't', 'vacuum did not increase frozen-page revision count after backend activity, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_frozen_pages', tab_value => $rev_all_frozen_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid) = $rev_all_visible_pages
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';"
+);
+ok($res eq 't', 'vacuum did not increase visible-page revision count after backend activity, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_visible_pages', tab_value => $rev_all_visible_pages,);
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 4286c266e17..e4a77878beb 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1844,7 +1844,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2266,7 +2268,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2321,9 +2325,43 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.missed_dead_pages,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.missed_dead_tuples,
+ stats.wraparound_failsafe,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 905f9bca959..62f2ac11659 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -139,4 +139,4 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
-test: tablespace
+test: tablespace
\ No newline at end of file
--
2.39.5 (Apple Git-154)
From d09c2f688fc8776b239a39f0cd9cda5488dba812 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 9 Dec 2025 10:56:54 +0300
Subject: [PATCH 2/5] Machinery for grabbing an extended vacuum statistics on
index relations.
They are gathered separatelly from table statistics.
As for tables, we gather vacuum shared buffers statistics for index relations like
value of total_blks_hit, total_blks_read, total_blks_dirtied, wal statistics, io time
during flushing buffer pages to disk, delay and total time.
Due to the fact that such statistics are common as for tables, as for indexes we
set them in the union ExtVacReport structure. We only added some determination 'type'
field to highlight what kind belong to these statistics: PGSTAT_EXTVAC_TABLE or
PGSTAT_EXTVAC_INDEX. Generally, PGSTAT_EXTVAC_INVALID type leads to wrong code process.
Some statistics belong only one type of both tables or indexes. So, we added substructures
sych table and index inside ExtVacReport structure.
Therefore, we gather only for tables such statistics like number of scanned, removed pages,
their charecteristics according VM (all-visible and frozen). In addition, for tables we
gather number frozen, deleted and recently dead tuples and how many times vacuum processed
indexes for tables.
Controversally for indexes we gather number of deleted pages and deleted tuples only.
As for tables, deleted pages and deleted tuples reflect the overall performance of the vacuum
for the index relationship.
Since the vacuum cleans up references to tuple indexes before cleaning up table tuples,
which adds some complexity to the vacuum process, namely the vacuum switches from cleaning up
a table to its indexes and back during its operation, we need to save the vacuum statistics
collected for the heap before it starts cleaning up the indexes.
That's why it's necessary to track the vacuum statistics for the heap several times during
the vacuum procedure. To avoid sending the statistics to the Cumulative Statistics System
several times, we save these statistics in the LVRelState structure and only after vacuum
finishes cleaning up the heap, it sends them to the Cumulative Statistics System.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 232 +++++++++++++----
src/backend/catalog/system_views.sql | 32 +++
src/backend/commands/vacuumparallel.c | 10 +
src/backend/utils/activity/pgstat_relation.c | 45 ++--
src/backend/utils/adt/pgstatfuncs.c | 92 ++++++-
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 25 ++
src/include/pgstat.h | 77 ++++--
.../vacuum-extending-in-repetable-read.out | 4 +-
.../t/050_vacuum_extending_basic_test.pl | 237 +++++++++++++++++-
src/test/regress/expected/rules.out | 22 ++
11 files changed, 681 insertions(+), 104 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 66e09d0a0cf..719ce90d96d 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -290,6 +290,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -412,6 +413,7 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to
* prevent anti-wraparound
* shutdown */
+ ExtVacReport extVacReportIdx;
} LVRelState;
@@ -423,19 +425,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-/*
- * Counters and usage data for extended stats tracking.
- */
-typedef struct LVExtStatCounters
-{
- TimestampTz starttime;
- WalUsage walusage;
- BufferUsage bufusage;
- double VacuumDelayTime;
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-} LVExtStatCounters;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static void heap_vacuum_eager_scan_setup(LVRelState *vacrel,
@@ -565,27 +554,25 @@ extvac_stats_end(Relation rel, LVExtStatCounters * counters,
endtime = GetCurrentTimestamp();
TimestampDifference(counters->starttime, endtime, &secs, &usecs);
- memset(report, 0, sizeof(ExtVacReport));
-
/*
* Fill additional statistics on a vacuum processing operation.
*/
- report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
- report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
- report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
- report->total_blks_written = bufusage.shared_blks_written;
+ report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written += bufusage.shared_blks_written;
- report->wal_records = walusage.wal_records;
- report->wal_fpi = walusage.wal_fpi;
- report->wal_bytes = walusage.wal_bytes;
+ report->wal_records += walusage.wal_records;
+ report->wal_fpi += walusage.wal_fpi;
+ report->wal_bytes += walusage.wal_bytes;
- report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
- report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
- report->total_time = secs * 1000. + usecs / 1000.;
+ report->total_time += secs * 1000. + usecs / 1000.;
if (!rel->pgstat_info || !pgstat_track_counts)
@@ -595,12 +582,122 @@ extvac_stats_end(Relation rel, LVExtStatCounters * counters,
*/
return;
- report->blks_fetched =
+ report->blks_fetched +=
rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
- report->blks_hit =
+ report->blks_hit +=
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ /* Set initial values for common heap and index statistics */
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, ExtVacReport * report)
+{
+ memset(report, 0, sizeof(ExtVacReport));
+
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacStats)
+{
+ /* Fill heap-specific extended stats fields */
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
+ extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
+ extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
+ extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
+ extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
+ extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
+ extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
+ extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
+ extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+
+ extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
+ extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+
+}
+
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacIdxStats)
+{
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
+ vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
+ vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
+ vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
+ vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
+ vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
+ vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
+ vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
+ vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
+ vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
+
+ vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -760,11 +857,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
char **indnames = NULL;
LVExtStatCounters extVacCounters;
ExtVacReport extVacReport;
- ExtVacReport allzero;
/* Initialize vacuum statistics */
- memset(&allzero, 0, sizeof(ExtVacReport));
- extVacReport = allzero;
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -820,6 +915,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
vac_open_indexes(vacrel->rel, RowExclusiveLock, &vacrel->nindexes,
@@ -1078,20 +1175,6 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Make generic extended vacuum stats report */
extvac_stats_end(rel, &extVacCounters, &extVacReport);
- /* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
- extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
- extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
- extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
- extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
-
/*
* Report results to the cumulative stats system, too.
*
@@ -1102,6 +1185,13 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
+
+ /*
+ * Make generic extended vacuum stats report and fill heap-specific
+ * extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
pgstat_report_vacuum(rel,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
@@ -2811,10 +2901,20 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
* that parallel VACUUM only gets the precheck and this postcheck.
@@ -3244,10 +3344,20 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
estimated_count);
+
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
/* Reset the progress counters */
@@ -3273,6 +3383,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3291,6 +3406,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3299,6 +3415,15 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(indrel,
+ 0, 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3323,6 +3448,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3342,12 +3472,22 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(indrel,
+ 0, 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ffb407d414f..47b6a00d297 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1502,3 +1502,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 114cd7c31d3..43450685b09 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,11 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(indrel,
+ 0, 0, 0, &extVacReport);
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 361713479e8..4bd6afc3794 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -1036,20 +1036,35 @@ pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
- dst->vm_new_visible_pages += src->vm_new_visible_pages;
- dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->missed_dead_pages += src->missed_dead_pages;
- dst->missed_dead_tuples += src->missed_dead_tuples;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->tuples_deleted += src->tuples_deleted;
+ }
+ }
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index d7dfda0c1a7..755751c3b46 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2360,18 +2360,19 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2393,3 +2394,72 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 915a5a7822f..e957781b623 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12630,4 +12630,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 6b997bc7fb1..b48ace6084b 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -300,6 +301,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -413,4 +434,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, ExtVacReport * report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 46d12fa3bd0..f2881dbb6f9 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -114,11 +114,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -155,23 +163,58 @@ typedef struct ExtVacReport
* point, in msec */
double total_time; /* total time of a vacuum operation, in msec */
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
- int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
- int64 vm_new_visible_frozen_pages; /* pages marked in VM as
- * all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to
- * failure to get a cleanup lock */
- int64 missed_dead_pages; /* pages with missed dead tuples */
int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still
- * visible to some transaction */
- int64 index_vacuum_count; /* the number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to
- * prevent anti-wraparound
- * shutdown */
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by
+ * VM) */
+ int64 pages_removed; /* heap pages removed by vacuum
+ * "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as
+ * all-visible */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are
+ * still visible to some
+ * transaction */
+ int64 vm_new_frozen_pages; /* pages marked in VM as
+ * frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as
+ * all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as
+ * all-visible and
+ * frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due
+ * to failure to get a cleanup
+ * lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency
+ * vacuums to prevent
+ * anti-wraparound
+ * shutdown */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */ ;
} ExtVacReport;
/* ----------
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 87f7e40b4a6..6d960423912 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -34,7 +34,7 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
(1 row)
step s1_commit: COMMIT;
@@ -48,6 +48,6 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
(1 row)
diff --git a/src/test/recovery/t/050_vacuum_extending_basic_test.pl b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
index 7e25a3fe63f..8f7b1e2909b 100644
--- a/src/test/recovery/t/050_vacuum_extending_basic_test.pl
+++ b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
@@ -2,9 +2,10 @@
# Test cumulative vacuum stats system using TAP
#
# This test validates the accuracy and behavior of cumulative vacuum statistics
-# across tables using:
+# across heap tables, indexes using:
#
# • pg_stat_vacuum_tables
+# • pg_stat_vacuum_indexes
#
# A polling helper function repeatedly checks the stats views until expected
# deltas appear or a configurable timeout expires. This guarantees that
@@ -62,7 +63,7 @@ $node->safe_psql($dbname, q{
$node->safe_psql(
$dbname,
- "CREATE TABLE vestat (x int)
+ "CREATE TABLE vestat (x int PRIMARY KEY)
WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
ANALYZE vestat;"
@@ -80,12 +81,15 @@ my $updated = 0;
#------------------------------------------------------------------------------
# wait_for_vacuum_stats
#
-# Polls pg_stat_vacuum_tables until the table-level counters exceed
-# the provided baselines, or until the configured timeout elapses.
+# Polls pg_stat_vacuum_tables and pg_stat_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
#
# Expected named args (baseline values):
# tab_tuples_deleted
# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
#
# Returns: 1 if the condition is met before timeout, 0 otherwise.
#------------------------------------------------------------------------------
@@ -94,6 +98,8 @@ sub wait_for_vacuum_stats {
my (%args) = @_;
my $tab_tuples_deleted = $args{tab_tuples_deleted} or 0;
my $tab_wal_records = $args{tab_wal_records} or 0;
+ my $idx_tuples_deleted = $args{idx_tuples_deleted} or 0;
+ my $idx_wal_records = $args{idx_wal_records} or 0;
my $start = time();
while ((time() - $start) < $timeout) {
@@ -101,9 +107,14 @@ sub wait_for_vacuum_stats {
my $result_query = $node->safe_psql(
$dbname,
"VACUUM vestat;
- SELECT tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
FROM pg_stat_vacuum_tables
- WHERE relname = 'vestat';"
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM pg_stat_vacuum_indexes
+ WHERE relname = 'vestat_pkey');"
);
return 1 if ($result_query eq 't');
@@ -126,6 +137,12 @@ my $wal_records = 0;
my $wal_bytes = 0;
my $wal_fpi = 0;
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
my $pages_frozen_prev = 0;
my $tuples_deleted_prev = 0;
my $pages_scanned_prev = 0;
@@ -134,11 +151,17 @@ my $wal_records_prev = 0;
my $wal_bytes_prev = 0;
my $wal_fpi_prev = 0;
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
#------------------------------------------------------------------------------
# fetch_vacuum_stats
#
-# Reads current values of relevant vacuum counters for the test table,
-# storing them in package variables for subsequent comparisons.
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
#------------------------------------------------------------------------------
sub fetch_vacuum_stats {
@@ -153,6 +176,18 @@ sub fetch_vacuum_stats {
$base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
($pages_frozen, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
= split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_indexes
+ WHERE relname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
}
#------------------------------------------------------------------------------
@@ -169,6 +204,12 @@ sub save_vacuum_stats {
$wal_records_prev = $wal_records;
$wal_bytes_prev = $wal_bytes;
$wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
}
#------------------------------------------------------------------------------
@@ -195,7 +236,20 @@ sub print_vacuum_stats_on_error {
" pages_removed = $pages_removed\n" .
" wal_records = $wal_records\n" .
" wal_bytes = $wal_bytes\n" .
- " wal_fpi = $wal_fpi\n"
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_removed = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_removed = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
);
};
@@ -203,7 +257,8 @@ sub print_vacuum_stats_on_error {
# fetch_vacuum_stats during mismatch
#
# Print current values and old values of relevant vacuum counters for the test
-# table, storing them in package variables for subsequent comparisons.
+# table and its primary index, storing them in package variables for subsequent
+# comparisons.
#------------------------------------------------------------------------------
sub fetch_error_base_tab_vacuum_statistics {
@@ -258,6 +313,54 @@ sub fetch_error_wal_tab_vacuum_statistics {
);
}
+sub fetch_error_base_idx_vacuum_statistics {
+
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted
+ FROM pg_stat_vacuum_indexes
+ WHERE relname = 'vestat_pkey';"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($cur_tuples_deleted, $cur_pages_deleted) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR INDEX:\n" .
+ " Baseline:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_removed = $index_pages_deleted\n" .
+ " Current:\n" .
+ " tuples_deleted = $cur_tuples_deleted\n" .
+ " pages_deleted = $cur_pages_deleted\n"
+ );
+}
+
+sub fetch_error_wal_idx_vacuum_statistics {
+
+ my $wal_raw = $node->safe_psql(
+ $dbname,
+ "SELECT wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_indexes
+ WHERE relname = 'vestat_pkey';"
+ );
+
+ $wal_raw =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($cur_wal_rec, $cur_wal_bytes, $cur_wal_fpi) = split /\s+/, $wal_raw;
+
+ diag(
+ "WAL STATS MISMATCH FOR INDEX:\n" .
+ " Baseline:\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n" .
+ " Current:\n" .
+ " wal_records = $cur_wal_rec\n" .
+ " wal_bytes = $cur_wal_bytes\n" .
+ " wal_fpi = $cur_wal_fpi\n"
+ );
+}
+
#------------------------------------------------------------------------------
# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
#------------------------------------------------------------------------------
@@ -270,7 +373,9 @@ $node->safe_psql($dbname, "VACUUM vestat;");
# Poll the stats view until expected deltas appear or timeout
$updated = wait_for_vacuum_stats(
tab_tuples_deleted => 0,
- tab_wal_records => 0
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
);
ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds after vacuuming half-deleted table";
@@ -290,6 +395,12 @@ ok($wal_records > $wal_records_prev, 'table wal_records has increased');
ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -307,6 +418,8 @@ $node->safe_psql($dbname, "VACUUM vestat;");
$updated = wait_for_vacuum_stats(
tab_tuples_deleted => $tuples_deleted_prev,
tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
);
ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
@@ -327,6 +440,12 @@ ok($wal_records > $wal_records_prev, 'table wal_records has increased');
ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -357,6 +476,12 @@ ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -379,6 +504,8 @@ $node->safe_psql(
$updated = wait_for_vacuum_stats(
tab_tuples_deleted => $tuples_deleted,
tab_wal_records => $wal_records,
+ idx_tuples_deleted => $index_tuples_deleted,
+ idx_wal_records => $index_wal_records,
);
ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
@@ -399,6 +526,12 @@ ok($wal_records > $wal_records_prev, 'table wal_records has increased');
ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -421,7 +554,9 @@ $node->safe_psql($dbname, "VACUUM vestat;");
$updated = wait_for_vacuum_stats(
tab_tuples_deleted => 0,
- tab_wal_records => $wal_records_prev
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
);
ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (tuples_deleted and wal_records advanced)')
@@ -442,6 +577,12 @@ ok($wal_records > $wal_records_prev, 'table wal_records has increased');
ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -464,7 +605,9 @@ $node->safe_psql(
$updated = wait_for_vacuum_stats(
tab_tuples_deleted => 0,
- tab_wal_records => $wal_records
+ tab_wal_records => $wal_records,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
);
ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (tuples_deleted and wal_records advanced)')
@@ -485,6 +628,12 @@ ok($wal_records > $wal_records_prev, 'table wal_records has increased');
ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -513,6 +662,34 @@ $base_stats = $node->safe_psql(
);
ok($base_stats eq 't', 'heap vacuum stats return from the current relation and database as expected');
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) = 1 FROM pg_stat_vacuum_indexes($dboid, $reloid);"
+);
+ok($base_stats eq 't', 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) = 0 FROM pg_stats_vacuum_tables($dboid, 1);"
+);
+ok($base_stats eq 't', 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) = 0 FROM pg_stat_vacuum_indexes($dboid, 1);"
+);
+ok($base_stats eq 't', 'index vacuum stats return no rows, as expected');
+
+
#------------------------------------------------------------------------------
# Test 9: Check relation-level vacuum statistics from another database
#------------------------------------------------------------------------------
@@ -523,6 +700,14 @@ $base_stats = $node->safe_psql(
FROM pg_stat_vacuum_tables
WHERE relname = 'vestat';"
);
+ok($base_stats eq 't', 'check the printing table vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 0
+ FROM pg_stat_vacuum_indexes
+ WHERE relname = 'vestat_pkey';"
+);
ok($base_stats eq 't', 'check the printing heap vacuum extended statistics from another database are not available');
$reloid = $node->safe_psql(
@@ -543,6 +728,23 @@ $base_stats = $node->safe_psql(
is($base_stats, 't', 'vacuum stats for common heap objects available');
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) = 1
+ FROM pg_stat_vacuum_indexes(0, $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
#------------------------------------------------------------------------------
# Test 11: Cleanup checks: ensure functions return empty sets for OID = 0
#------------------------------------------------------------------------------
@@ -562,6 +764,15 @@ $base_stats = $node->safe_psql(
);
ok($base_stats eq 't', 'pg_stat_vacuum_tables correctly returns no rows for OID = 0');
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*) = 0
+ FROM pg_stat_vacuum_indexes WHERE relid = 0;
+ }
+);
+ok($base_stats eq 't', 'pg_stat_vacuum_indexes correctly returns no rows for OID = 0');
+
$node->safe_psql('postgres',
"DROP DATABASE $dbname;"
);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index e4a77878beb..7e6029394cb 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2330,6 +2330,28 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schemaname,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
rel.relname,
stats.relid,
--
2.39.5 (Apple Git-154)
From 2051db2600566dc88040cee97868469aa35440d7 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 1 Sep 2025 21:43:33 +0300
Subject: [PATCH 3/5] Machinery for grabbing an extended vacuum statistics on
databases.
Database vacuum statistics information is the collected general
vacuum statistics indexes and tables owned by the databases, which
they belong to.
In addition to the fact that there are far fewer databases in a system
than relations, vacuum statistics for a database contain fewer statistics
than relations, but they are enough to indicate that something may be
wrong in the system and prompt the administrator to enable extended
monitoring for relations.
So, buffer, wal, statistics of I/O time of read and writen blocks
statistics will be observed because they are collected for both
tables, indexes. In addition, we show the number of errors caught
during operation of the vacuum only for the error level.
wraparound_failsafe_count is a number of times when the vacuum starts
urgent cleanup to prevent wraparound problem which is critical for
the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 2 +-
src/backend/catalog/system_views.sql | 26 +++++++-
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 13 +++-
src/backend/utils/adt/pgstatfuncs.c | 62 ++++++++++++++++++-
src/include/catalog/pg_proc.dat | 13 +++-
src/include/pgstat.h | 7 +--
.../vacuum-extending-in-repetable-read.spec | 6 ++
.../t/050_vacuum_extending_basic_test.pl | 49 ++++++++++++---
.../t/051_vacuum_extending_freeze_test.pl | 48 +++++---------
src/test/regress/expected/rules.out | 17 +++++
11 files changed, 195 insertions(+), 49 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 719ce90d96d..fcd92a43dda 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -663,7 +663,7 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacStats
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 47b6a00d297..dc86b1ee212 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1533,4 +1533,28 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.wraparound_failsafe AS wraparound_failsafe
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index b31f20d41bc..65207d30378 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -485,6 +485,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 4bd6afc3794..2675c541369 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -215,6 +215,7 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -273,6 +274,16 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
}
/*
@@ -1032,6 +1043,7 @@ pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
dst->blk_write_time += src->blk_write_time;
dst->delay_time += src->delay_time;
dst->total_time += src->total_time;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
if (!accumulate_reltype_specific_info)
return;
@@ -1059,7 +1071,6 @@ pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
dst->table.index_vacuum_count += src->table.index_vacuum_count;
dst->table.missed_dead_pages += src->table.missed_dead_pages;
dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
}
else if (dst->type == PGSTAT_EXTVAC_INDEX)
{
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 755751c3b46..4e2714f2e6a 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2371,7 +2371,7 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
@@ -2463,3 +2463,63 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index e957781b623..c3a2adb96f1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12631,12 +12631,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,wraparound_failsafe}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f2881dbb6f9..f3bdc1c38df 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -165,6 +165,9 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
+ int32 wraparound_failsafe_count; /* the number of times to prevent
+ * wraparound problem */
+
ExtVacReportType type; /* heap, index, etc. */
/* ----------
@@ -205,10 +208,6 @@ typedef struct ExtVacReport
* lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency
- * vacuums to prevent
- * anti-wraparound
- * shutdown */
} table;
struct
{
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5893d89573d..cfec3159580 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -18,6 +18,9 @@ teardown
}
session s1
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s1_begin_repeatable_read {
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
select count(ival) from test_vacuum_stat_isolation where id>900;
@@ -25,6 +28,9 @@ step s1_begin_repeatable_read {
step s1_commit { COMMIT; }
session s2
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
diff --git a/src/test/recovery/t/050_vacuum_extending_basic_test.pl b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
index 8f7b1e2909b..bd3cb544e30 100644
--- a/src/test/recovery/t/050_vacuum_extending_basic_test.pl
+++ b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
@@ -2,10 +2,11 @@
# Test cumulative vacuum stats system using TAP
#
# This test validates the accuracy and behavior of cumulative vacuum statistics
-# across heap tables, indexes using:
+# across heap tables, indexes, and databases using:
#
# • pg_stat_vacuum_tables
# • pg_stat_vacuum_indexes
+# • pg_stat_vacuum_database
#
# A polling helper function repeatedly checks the stats views until expected
# deltas appear or a configurable timeout expires. This guarantees that
@@ -672,20 +673,20 @@ $reloid = $node->safe_psql(
# Check if we can get vacuum statistics of particular index relation in the current database
$base_stats = $node->safe_psql(
$dbname,
- "SELECT count(*) = 1 FROM pg_stat_vacuum_indexes($dboid, $reloid);"
+ "SELECT count(*) = 1 FROM pg_stat_get_vacuum_indexes($reloid);"
);
ok($base_stats eq 't', 'index vacuum stats return from the current relation and database as expected');
# Check if we return empty results if vacuum statistics with particular oid doesn't exist
$base_stats = $node->safe_psql(
$dbname,
- "SELECT count(*) = 0 FROM pg_stats_vacuum_tables($dboid, 1);"
+ "SELECT count(*) = 0 FROM pg_stat_get_vacuum_tables(1);"
);
ok($base_stats eq 't', 'table vacuum stats return no rows, as expected');
$base_stats = $node->safe_psql(
$dbname,
- "SELECT count(*) = 0 FROM pg_stat_vacuum_indexes($dboid, 1);"
+ "SELECT count(*) = 0 FROM pg_stat_get_vacuum_indexes(1);"
);
ok($base_stats eq 't', 'index vacuum stats return no rows, as expected');
@@ -708,7 +709,31 @@ $base_stats = $node->safe_psql(
FROM pg_stat_vacuum_indexes
WHERE relname = 'vestat_pkey';"
);
-ok($base_stats eq 't', 'check the printing heap vacuum extended statistics from another database are not available');
+ok($base_stats eq 't', 'check the printing index vacuum extended statistics from another database are not available');
+
+#--------------------------------------------------------------------------------------
+# Test 10: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit > 0 AND total_blks_dirtied > 0
+ AND total_blks_written > 0 AND wal_records > 0
+ AND wal_fpi > 0 AND wal_bytes > 0
+ FROM pg_stat_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = pg_stat_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from the current database are available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) > 0
+ FROM pg_stat_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = pg_stat_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
$reloid = $node->safe_psql(
$dbname,
@@ -739,7 +764,7 @@ $base_stats = $node->safe_psql(
$dbname,
qq{
SELECT count(*) = 1
- FROM pg_stat_vacuum_indexes(0, $indoid);
+ FROM pg_stat_get_vacuum_indexes($indoid);
}
);
@@ -773,8 +798,18 @@ $base_stats = $node->safe_psql(
);
ok($base_stats eq 't', 'pg_stat_vacuum_indexes correctly returns no rows for OID = 0');
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT COUNT(*) = 0
+ FROM pg_stat_vacuum_database WHERE dboid = 0;
+ }
+);
+ok($base_stats eq 't', 'pg_stat_vacuum_database correctly returns no rows for OID = 0');
+
$node->safe_psql('postgres',
- "DROP DATABASE $dbname;"
+ "DROP DATABASE $dbname;
+ VACUUM;"
);
$node->stop;
diff --git a/src/test/recovery/t/051_vacuum_extending_freeze_test.pl b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
index a9b5d6cb739..7528f20098b 100644
--- a/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
+++ b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
@@ -91,11 +91,17 @@ sub wait_for_vacuum_stats {
my $start = time();
my $sql;
+ my $vacuum_run = 0;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ $vacuum_run = 1;
+ }
while ((time() - $start) < $timeout) {
if ($run_vacuum) {
- $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
$sql = "
SELECT ($tab_frozen_column > $tab_all_frozen_pages_count AND
$tab_visible_column > $tab_all_visible_pages_count)
@@ -213,20 +219,6 @@ $node->safe_psql($dbname, q{
VACUUM (FREEZE, VERBOSE) vestat;
});
-# Poll the stats view until the expected deltas appear or timeout.
-# We do not expect rev_all_* counters to change here, so we pass -1 for them.
-$updated = wait_for_vacuum_stats(
- tab_frozen_column => 'vm_new_frozen_pages',
- tab_visible_column => 'vm_new_visible_pages',
- tab_all_frozen_pages_count => 0,
- tab_all_visible_pages_count => 0,
- run_vacuum => 1,
-);
-
-ok($updated,
- 'vacuum stats updated after vacuuming the table (vm_new_frozen_pages and vm_new_visible_pages advanced)')
- or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
-
#------------------------------------------------------------------------------
# Snapshot current statistics for later comparison
#------------------------------------------------------------------------------
@@ -238,7 +230,7 @@ fetch_vacuum_stats();
#------------------------------------------------------------------------------
$res = $node->safe_psql($dbname, q{
- SELECT vm_new_frozen_pages > 0 FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ SELECT vm_new_frozen_pages = 0 FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
});
ok($res eq 't', 'vacuum froze some pages, as expected') or
fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_frozen_pages', tab_value => $vm_new_frozen_pages,);
@@ -335,32 +327,24 @@ fetch_vacuum_stats();
$node->safe_psql($dbname, q{ VACUUM (FREEZE, VERBOSE) vestat; });
-# Poll until stats update or timeout.
-# We pass current snapshot values for vm_new_frozen_pages/vm_new_visible_pages and expect rev counters unchanged.
-$updated = wait_for_vacuum_stats(
- tab_frozen_column => 'vm_new_frozen_pages',
- tab_visible_column => 'vm_new_visible_pages',
- tab_all_frozen_pages_count => $vm_new_frozen_pages,
- tab_all_visible_pages_count => $vm_new_visible_pages,
- run_vacuum => 1,
-);
-
-ok($updated,
- 'vacuum stats updated after vacuuming the all-updated table (vm_new_frozen_pages and vm_new_visible_pages advanced)')
- or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
-
#------------------------------------------------------------------------------
# Verify statistics after final vacuum
# Check updated stats after backend work
#------------------------------------------------------------------------------
+
+# Fetch updated statistics to get the new baseline for comparison
+my $old_vm_new_frozen_pages = $vm_new_frozen_pages;
+my $old_vm_new_visible_pages = $vm_new_visible_pages;
+fetch_vacuum_stats();
+
$res = $node->safe_psql($dbname,
- "SELECT vm_new_frozen_pages > $vm_new_frozen_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+ "SELECT vm_new_frozen_pages = $old_vm_new_frozen_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
);
ok($res eq 't', 'vacuum froze some pages after backend activity, as expected') or
fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_frozen_pages', tab_value => $vm_new_frozen_pages,);
$res = $node->safe_psql($dbname,
- "SELECT vm_new_visible_pages > $vm_new_visible_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+ "SELECT vm_new_visible_pages > $old_vm_new_visible_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
);
ok($res eq 't', 'vacuum marked pages all-visible after backend activity, as expected') or
fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_visible_pages', tab_value => $vm_new_visible_pages,);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 7e6029394cb..b627c85e332 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2330,6 +2330,23 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schemaname,
rel.relname,
--
2.39.5 (Apple Git-154)
From 12f4596c25e15d1f7566b87334615ad112cfb64e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 21 Dec 2025 01:40:06 +0300
Subject: [PATCH 4/5] Vacuum statistics have been separated from regular
relation and database statistics to reduce memory usage. Dedicated
PGSTAT_KIND_VACUUM_RELATION and PGSTAT_KIND_VACUUM_DB entries were added to
the stats collector to efficiently allocate memory for vacuum-specific
metrics, which require significantly more space per relation.
---
src/backend/access/heap/vacuumlazy.c | 124 +++++-----
src/backend/catalog/heap.c | 1 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/system_views.sql | 177 ++++++++-------
src/backend/commands/dbcommands.c | 1 +
src/backend/commands/vacuumparallel.c | 5 +-
src/backend/utils/activity/Makefile | 1 +
src/backend/utils/activity/pgstat.c | 30 ++-
src/backend/utils/activity/pgstat_database.c | 10 +-
src/backend/utils/activity/pgstat_relation.c | 75 +-----
src/backend/utils/activity/pgstat_vacuum.c | 214 ++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 149 ++++++------
src/backend/utils/misc/guc_parameters.dat | 6 +
src/include/commands/vacuum.h | 2 +-
src/include/pgstat.h | 208 +++++++++--------
src/include/utils/pgstat_internal.h | 15 ++
src/include/utils/pgstat_kind.h | 4 +-
.../t/050_vacuum_extending_basic_test.pl | 21 +-
.../t/051_vacuum_extending_freeze_test.pl | 1 +
src/test/regress/expected/rules.out | 146 ++++++------
20 files changed, 735 insertions(+), 456 deletions(-)
create mode 100644 src/backend/utils/activity/pgstat_vacuum.c
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index fcd92a43dda..3f1ed040908 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -413,7 +413,8 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to
* prevent anti-wraparound
* shutdown */
- ExtVacReport extVacReportIdx;
+
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -505,6 +506,9 @@ extvac_stats_start(Relation rel, LVExtStatCounters * counters)
{
TimestampTz starttime;
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
memset(counters, 0, sizeof(LVExtStatCounters));
starttime = GetCurrentTimestamp();
@@ -536,7 +540,7 @@ extvac_stats_start(Relation rel, LVExtStatCounters * counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters * counters,
- ExtVacReport * report)
+ PgStat_CommonCounts * report)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -544,6 +548,11 @@ extvac_stats_end(Relation rel, LVExtStatCounters * counters,
long secs;
int usecs;
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+
/* Calculate diffs of global stat parameters on WAL and buffer usage. */
memset(&walusage, 0, sizeof(WalUsage));
WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
@@ -592,6 +601,9 @@ void
extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx * counters)
{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
/* Set initial values for common heap and index statistics */
extvac_stats_start(rel, &counters->common);
counters->pages_deleted = counters->tuples_removed = 0;
@@ -609,11 +621,15 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
void
extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx * counters, ExtVacReport * report)
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
{
- memset(report, 0, sizeof(ExtVacReport));
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ extvac_stats_end(rel, &counters->common, &report->common);
- extvac_stats_end(rel, &counters->common, report);
report->type = PGSTAT_EXTVAC_INDEX;
if (stats != NULL)
@@ -624,7 +640,7 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
*/
/* Fill index-specific extended stats fields */
- report->tuples_deleted =
+ report->common.tuples_deleted =
stats->tuples_removed - counters->tuples_removed;
report->index.pages_deleted =
stats->pages_deleted - counters->pages_deleted;
@@ -647,8 +663,11 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
* procudure.
*/
static void
-accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacStats)
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
/* Fill heap-specific extended stats fields */
extVacStats->type = PGSTAT_EXTVAC_TABLE;
extVacStats->table.pages_scanned = vacrel->scanned_pages;
@@ -656,46 +675,45 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacStats
extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
- extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
- extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
- extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
- extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
- extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
- extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
- extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
- extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
- extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
- extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
}
static void
-accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacIdxStats)
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacIdxStats)
{
/* Fill heap-specific extended stats fields */
- vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
- vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
- vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
- vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
- vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
- vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
- vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
- vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
- vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
- vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
-
- vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
}
@@ -856,10 +874,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ErrorContextCallback errcallback;
char **indnames = NULL;
LVExtStatCounters extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Initialize vacuum statistics */
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -915,7 +933,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
- memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+ memset(&vacrel->extVacReportIdx, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
@@ -1173,7 +1192,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
&frozenxid_updated, &minmulti_updated, false);
/* Make generic extended vacuum stats report */
- extvac_stats_end(rel, &extVacCounters, &extVacReport);
+ /* extvac_stats_end(rel, &extVacCounters, &extVacReport.common); */
/*
* Report results to the cumulative stats system, too.
@@ -1190,14 +1209,14 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* Make generic extended vacuum stats report and fill heap-specific
* extended stats fields.
*/
- extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport.common);
accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+ pgstat_report_vacuum_extstats(vacrel->reloid, rel->rd_rel->relisshared, &extVacReport);
pgstat_report_vacuum(rel,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime,
- &extVacReport);
+ starttime);
pgstat_progress_end_command();
if (instrument)
@@ -2902,9 +2921,9 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -2912,7 +2931,7 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
- extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
/*
@@ -3345,9 +3364,9 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -3356,7 +3375,7 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
vacrel->num_index_scans,
estimated_count);
- extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
@@ -3384,7 +3403,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3421,8 +3443,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
if (!ParallelVacuumIsActive(vacrel))
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(indrel,
- 0, 0, 0, &extVacReport);
+ pgstat_report_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -3449,7 +3470,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3485,8 +3506,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
if (!ParallelVacuumIsActive(vacrel))
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(indrel,
- 0, 0, 0, &extVacReport);
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared, &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 265cc3e5fbf..680b76b8ef9 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1883,6 +1883,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8dea58ad96b..e906f9e1856 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2327,6 +2327,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index dc86b1ee212..4ec2a7d9f10 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1462,99 +1462,104 @@ GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
--
CREATE VIEW pg_stat_vacuum_tables AS
-SELECT
- ns.nspname AS schemaname,
- rel.relname AS relname,
- stats.relid as relid,
-
- stats.total_blks_read AS total_blks_read,
- stats.total_blks_hit AS total_blks_hit,
- stats.total_blks_dirtied AS total_blks_dirtied,
- stats.total_blks_written AS total_blks_written,
-
- stats.rel_blks_read AS rel_blks_read,
- stats.rel_blks_hit AS rel_blks_hit,
-
- stats.pages_scanned AS pages_scanned,
- stats.pages_removed AS pages_removed,
- stats.vm_new_frozen_pages AS vm_new_frozen_pages,
- stats.vm_new_visible_pages AS vm_new_visible_pages,
- stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
- stats.missed_dead_pages AS missed_dead_pages,
- stats.tuples_deleted AS tuples_deleted,
- stats.tuples_frozen AS tuples_frozen,
- stats.recently_dead_tuples AS recently_dead_tuples,
- stats.missed_dead_tuples AS missed_dead_tuples,
-
- stats.wraparound_failsafe AS wraparound_failsafe,
- stats.index_vacuum_count AS index_vacuum_count,
- stats.wal_records AS wal_records,
- stats.wal_fpi AS wal_fpi,
- stats.wal_bytes AS wal_bytes,
-
- stats.blk_read_time AS blk_read_time,
- stats.blk_write_time AS blk_write_time,
-
- stats.delay_time AS delay_time,
- stats.total_time AS total_time
-
-FROM pg_class rel
- JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
- LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
-WHERE rel.relkind = 'r';
+ SELECT
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ S.relid as relid,
+
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
+
+ S.pages_scanned AS pages_scanned,
+ S.pages_removed AS pages_removed,
+ S.vm_new_frozen_pages AS vm_new_frozen_pages,
+ S.vm_new_visible_pages AS vm_new_visible_pages,
+ S.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ S.missed_dead_pages AS missed_dead_pages,
+ S.tuples_deleted AS tuples_deleted,
+ S.tuples_frozen AS tuples_frozen,
+ S.recently_dead_tuples AS recently_dead_tuples,
+ S.missed_dead_tuples AS missed_dead_tuples,
+
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.index_vacuum_count AS index_vacuum_count,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+
+ FROM pg_class C JOIN
+ pg_namespace N ON N.oid = C.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(C.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
CREATE VIEW pg_stat_vacuum_indexes AS
-SELECT
- rel.oid as relid,
- ns.nspname AS schemaname,
- rel.relname AS relname,
+ SELECT
+ C.oid AS relid,
+ I.oid AS indexrelid,
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ I.relname AS indexrelname,
- total_blks_read AS total_blks_read,
- total_blks_hit AS total_blks_hit,
- total_blks_dirtied AS total_blks_dirtied,
- total_blks_written AS total_blks_written,
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
- rel_blks_read AS rel_blks_read,
- rel_blks_hit AS rel_blks_hit,
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
- pages_deleted AS pages_deleted,
- tuples_deleted AS tuples_deleted,
+ S.pages_deleted AS pages_deleted,
+ S.tuples_deleted AS tuples_deleted,
- wal_records AS wal_records,
- wal_fpi AS wal_fpi,
- wal_bytes AS wal_bytes,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
- blk_read_time AS blk_read_time,
- blk_write_time AS blk_write_time,
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
- delay_time AS delay_time,
- total_time AS total_time
-FROM
- pg_class rel
- JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
- LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+ FROM
+ pg_class C JOIN
+ pg_index X ON C.oid = X.indrelid JOIN
+ pg_class I ON I.oid = X.indexrelid
+ LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace),
+ LATERAL pg_stat_get_vacuum_indexes(I.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
CREATE VIEW pg_stat_vacuum_database AS
-SELECT
- db.oid as dboid,
- db.datname AS dbname,
-
- stats.db_blks_read AS db_blks_read,
- stats.db_blks_hit AS db_blks_hit,
- stats.total_blks_dirtied AS total_blks_dirtied,
- stats.total_blks_written AS total_blks_written,
-
- stats.wal_records AS wal_records,
- stats.wal_fpi AS wal_fpi,
- stats.wal_bytes AS wal_bytes,
-
- stats.blk_read_time AS blk_read_time,
- stats.blk_write_time AS blk_write_time,
-
- stats.delay_time AS delay_time,
- stats.total_time AS total_time,
- stats.wraparound_failsafe AS wraparound_failsafe
-FROM
- pg_database db,
- LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
+ SELECT
+ D.oid as dboid,
+ D.datname AS dbname,
+
+ S.db_blks_read AS db_blks_read,
+ S.db_blks_hit AS db_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time,
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.errors AS errors
+ FROM
+ pg_database D,
+ LATERAL pg_stat_get_vacuum_database(D.oid) S;
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index d1f3be89b35..bf3cd3b1cc9 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1815,6 +1815,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 43450685b09..c7dd2bb52f6 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,7 +869,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -911,8 +911,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
/* Make extended vacuum stats report for index */
extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(indrel,
- 0, 0, 0, &extVacReport);
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared, &extVacReport);
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 9c2443e1ecd..183f7514d2d 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index f317c6e8e90..cdc9cab01cf 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = false;
/* ----------
* state shared with pgstat_*.c
@@ -482,6 +482,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 65207d30378..80e6c7c229a 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
@@ -485,7 +494,6 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
- memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 2675c541369..e8665d23099 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
- bool accumulate_reltype_specific_info);
/*
@@ -210,12 +208,11 @@ pgstat_drop_relation(Relation rel)
*/
void
pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
- PgStat_Counter deadtuples, TimestampTz starttime, ExtVacReport * params)
+ PgStat_Counter deadtuples, TimestampTz starttime)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
- PgStatShared_Database *dbentry;
Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -237,8 +234,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
- pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
-
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -274,16 +269,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
-
- if (dboid != InvalidOid)
- {
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dboid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
-
- pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
- pgstat_unlock_entry(entry_ref);
- }
}
/*
@@ -918,6 +903,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
@@ -1027,55 +1018,3 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
-
-static void
-pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
- bool accumulate_reltype_specific_info)
-{
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->total_time += src->total_time;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_TABLE)
- {
- dst->table.pages_scanned += src->table.pages_scanned;
- dst->table.pages_removed += src->table.pages_removed;
- dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
- dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
- dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->table.tuples_frozen += src->table.tuples_frozen;
- dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
- dst->table.index_vacuum_count += src->table.index_vacuum_count;
- dst->table.missed_dead_pages += src->table.missed_dead_pages;
- dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->tuples_deleted += src->tuples_deleted;
- }
- }
-}
diff --git a/src/backend/utils/activity/pgstat_vacuum.c b/src/backend/utils/activity/pgstat_vacuum.c
new file mode 100644
index 00000000000..340ee24f26a
--- /dev/null
+++ b/src/backend/utils/activity/pgstat_vacuum.c
@@ -0,0 +1,214 @@
+#include "postgres.h"
+
+#include "pgstat.h"
+#include "utils/pgstat_internal.h"
+#include "utils/memutils.h"
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+bool pgstat_track_vacuum_statistics_for_relations = false;
+
+#define ACCUMULATE_FIELD(field) dst->field += src->field;
+
+#define ACCUMULATE_SUBFIELD(substruct, field) \
+ (dst->substruct.field += src->substruct.field)
+
+static void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUMULATE_FIELD(total_blks_read);
+ ACCUMULATE_FIELD(total_blks_hit);
+ ACCUMULATE_FIELD(total_blks_dirtied);
+ ACCUMULATE_FIELD(total_blks_written);
+
+ ACCUMULATE_FIELD(blks_fetched);
+ ACCUMULATE_FIELD(blks_hit);
+
+ ACCUMULATE_FIELD(wal_records);
+ ACCUMULATE_FIELD(wal_fpi);
+ ACCUMULATE_FIELD(wal_bytes);
+
+ ACCUMULATE_FIELD(blk_read_time);
+ ACCUMULATE_FIELD(blk_write_time);
+ ACCUMULATE_FIELD(delay_time);
+ ACCUMULATE_FIELD(total_time);
+
+ ACCUMULATE_FIELD(tuples_deleted);
+ ACCUMULATE_FIELD(wraparound_failsafe_count);
+}
+
+static void
+pgstat_accumulate_extvac_stats_relations(PgStat_VacuumRelationCounts * dst, PgStat_VacuumRelationCounts * src)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB && src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ ACCUMULATE_SUBFIELD(common, blks_fetched);
+ ACCUMULATE_SUBFIELD(common, blks_hit);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(table, pages_scanned);
+ ACCUMULATE_SUBFIELD(table, pages_removed);
+ ACCUMULATE_SUBFIELD(table, vm_new_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, tuples_frozen);
+ ACCUMULATE_SUBFIELD(table, recently_dead_tuples);
+ ACCUMULATE_SUBFIELD(table, index_vacuum_count);
+ ACCUMULATE_SUBFIELD(table, missed_dead_pages);
+ ACCUMULATE_SUBFIELD(table, missed_dead_tuples);
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(index, pages_deleted);
+ }
+}
+
+static void
+pgstat_accumulate_extvac_stats_db(PgStat_VacuumDBCounts * dst, PgStat_VacuumDBCounts * src)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+}
+
+/*
+ * Report that the table was just vacuumed and flush statistics.
+ */
+void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_VacuumRelation *shtabentry;
+ PgStatShared_VacuumDB *shdbentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_RELATION,
+ dboid, tableoid, false);
+ shtabentry = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+ pgstat_accumulate_extvac_stats_relations(&shtabentry->stats, params);
+
+ pgstat_unlock_entry(entry_ref);
+
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_DB,
+ dboid, InvalidOid, false);
+
+ shdbentry = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ pgstat_accumulate_common(&shdbentry->stats.common, ¶ms->common);
+
+ pgstat_unlock_entry(entry_ref);
+}
+
+/*
+ * Flush out pending stats for the entry
+ *
+ * If nowait is true, this function returns false if lock could not
+ * immediately acquired, otherwise true is returned.
+ */
+bool
+pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumRelation *shtabstats;
+ PgStat_RelationVacuumPending *pendingent; /* table entry of shared stats */
+
+ pendingent = (PgStat_RelationVacuumPending *) entry_ref->pending;
+ shtabstats = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts.
+ */
+ if (pg_memory_is_all_zeros(&pendingent,
+ sizeof(struct PgStat_RelationVacuumPending)))
+ return true;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ {
+ return false;
+ }
+
+ pgstat_accumulate_extvac_stats_relations(&(shtabstats->stats), &(pendingent->counts));
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the vacuum collected statistics for one relation or NULL.
+ */
+PgStat_VacuumRelationCounts *
+pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid)
+{
+ return (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_RELATION, dbid, relid);
+}
+
+PgStat_VacuumDBCounts *
+pgstat_fetch_stat_vacuum_dbentry(Oid dbid)
+{
+ return (PgStat_VacuumDBCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_DB, dbid, InvalidOid);
+}
+
+bool
+pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumDB *sharedent;
+ PgStat_VacuumDBCounts *pendingent;
+
+ pendingent = (PgStat_VacuumDBCounts *) entry_ref->pending;
+ sharedent = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ /* The entry was successfully flushed, add the same to database stats */
+ pgstat_accumulate_extvac_stats_db(&(sharedent->stats), pendingent);
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Find or create a local PgStat_VacuumDBCounts entry for dboid.
+ */
+PgStat_VacuumDBCounts *
+pgstat_prep_vacuum_database_pending(Oid dboid)
+{
+ PgStat_EntryRef *entry_ref;
+
+ /*
+ * This should not report stats on database objects before having
+ * connected to a database.
+ */
+ Assert(!OidIsValid(dboid) || OidIsValid(MyDatabaseId));
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_VACUUM_DB, dboid, InvalidOid,
+ NULL);
+
+ if (entry_ref == NULL)
+ return NULL;
+
+ return entry_ref->pending;
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 4e2714f2e6a..0a64f034a3f 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2314,7 +2314,6 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
-
/*
* Get the vacuum statistics for the heap tables.
*/
@@ -2324,41 +2323,45 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
char buf[256];
int i = 0;
+ /* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
- tabentry = pgstat_fetch_stat_tabentry(relid);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- if (!tabentry)
- {
- InitMaterializedSRF(fcinfo, 0);
- PG_RETURN_VOID();
- }
- else
+ if (!pending)
{
- extvacuum = &(tabentry->vacuum_ext);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, 0);
+
+ if (!pending)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
}
+ extvacuum = pending;
+
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
@@ -2366,28 +2369,28 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
@@ -2404,8 +2407,8 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
@@ -2415,48 +2418,51 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
- tabentry = pgstat_fetch_stat_tabentry(relid);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- if (tabentry == NULL)
- {
- InitMaterializedSRF(fcinfo, 0);
- PG_RETURN_VOID();
- }
- else
+ if (!pending)
{
- extvacuum = &(tabentry->vacuum_ext);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, 0);
+
+ if (!pending)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
}
+ extvacuum = pending;
+
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
@@ -2470,8 +2476,8 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
Oid dbid = PG_GETARG_OID(0);
- PgStat_StatDBEntry *dbentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumDBCounts *extvacuum;
+ PgStat_VacuumDBCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
@@ -2481,42 +2487,41 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
- dbentry = pgstat_fetch_stat_dbentry(dbid);
+ pending = pgstat_fetch_stat_vacuum_dbentry(dbid);
- if (dbentry == NULL)
+ if (!pending)
{
InitMaterializedSRF(fcinfo, 0);
PG_RETURN_VOID();
}
- else
- {
- extvacuum = &(dbentry->vacuum_ext);
- }
+
+ extvacuum = pending;
i = 0;
values[i++] = ObjectIdGetDatum(dbid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->errors);
Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 3b9d8349078..631df3a57c3 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -3084,6 +3084,12 @@
boot_val => 'false',
},
+{ name => 'track_vacuum_statistics', type => 'bool', context => 'PGC_SUSET', group => 'STATS_CUMULATIVE',
+ short_desc => 'Collects vacuum statistics for vacuum activity.',
+ variable => 'pgstat_track_vacuum_statistics',
+ boot_val => 'false',
+},
+
{ name => 'track_wal_io_timing', type => 'bool', context => 'PGC_SUSET', group => 'STATS_CUMULATIVE',
short_desc => 'Collects timing statistics for WAL I/O activity.',
variable => 'track_wal_io_timing',
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index b48ace6084b..6e85b08aa89 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -437,5 +437,5 @@ extern double anl_get_next_S(double t, int n, double *stateptr);
extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx * counters);
extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx * counters, ExtVacReport * report);
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f3bdc1c38df..61d488f1bf8 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -119,54 +119,100 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_TABLE = 1,
- PGSTAT_EXTVAC_INDEX = 2
-} ExtVacReportType;
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
/* ----------
+ * PgStat_TableCounts The actual per-table counts kept by a backend
*
- * ExtVacReport
+ * This struct should contain only actual event counters, because we make use
+ * of pg_memory_is_all_zeros() to detect whether there are any stats updates
+ * to apply.
*
- * Additional statistics of vacuum processing over a relation.
- * pages_removed is the amount by which the physically shrank,
- * if any (ie the change in its total size on disk)
- * pages_deleted refer to free space within the index file
+ * It is a component of PgStat_TableStatus (within-backend state).
+ *
+ * Note: for a table, tuples_returned is the number of tuples successfully
+ * fetched by heap_getnext, while tuples_fetched is the number of tuples
+ * successfully fetched by heap_fetch under the control of bitmap indexscans.
+ * For an index, tuples_returned is the number of index entries returned by
+ * the index AM, while tuples_fetched is the number of tuples successfully
+ * fetched by heap_fetch under the control of simple indexscans for this index.
+ *
+ * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
+ * actions, regardless of whether the transaction committed. delta_live_tuples,
+ * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
+ * Note that delta_live_tuples and delta_dead_tuples can be negative!
* ----------
*/
-typedef struct ExtVacReport
+typedef struct PgStat_TableCounts
{
- /*
- * number of blocks missed, hit, dirtied and written during a vacuum of
- * specific relation
- */
+ PgStat_Counter numscans;
+
+ PgStat_Counter tuples_returned;
+ PgStat_Counter tuples_fetched;
+
+ PgStat_Counter tuples_inserted;
+ PgStat_Counter tuples_updated;
+ PgStat_Counter tuples_deleted;
+ PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_newpage_updated;
+ bool truncdropped;
+
+ PgStat_Counter delta_live_tuples;
+ PgStat_Counter delta_dead_tuples;
+ PgStat_Counter changed_tuples;
+
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+} PgStat_TableCounts;
+
+typedef struct PgStat_CommonCounts
+{
+ /* blocks */
int64 total_blks_read;
int64 total_blks_hit;
int64 total_blks_dirtied;
int64 total_blks_written;
- /*
- * blocks missed and hit for just the heap during a vacuum of specific
- * relation
- */
+ /* heap blocks */
int64 blks_fetched;
int64 blks_hit;
- /* Vacuum WAL usage stats */
- int64 wal_records; /* wal usage: number of WAL records */
- int64 wal_fpi; /* wal usage: number of WAL full page images
- * produced */
- uint64 wal_bytes; /* wal usage: size of WAL records produced */
+ /* WAL */
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
- /* Time stats. */
- double blk_read_time; /* time spent reading pages, in msec */
- double blk_write_time; /* time spent writing pages, in msec */
- double delay_time; /* how long vacuum slept in vacuum delay
- * point, in msec */
- double total_time; /* total time of a vacuum operation, in msec */
+ /* Time */
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
- int64 tuples_deleted; /* tuples deleted by vacuum */
+ /* tuples */
+ int64 tuples_deleted;
- int32 wraparound_failsafe_count; /* the number of times to prevent
- * wraparound problem */
+ /* failsafe */
+ int32 wraparound_failsafe_count;
+} PgStat_CommonCounts;
+
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
ExtVacReportType type; /* heap, index, etc. */
@@ -185,6 +231,13 @@ typedef struct ExtVacReport
{
struct
{
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are
+ * still visible to some
+ * transaction */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due
+ * to failure to get a cleanup
+ * lock */
int64 pages_scanned; /* heap pages examined (not skipped by
* VM) */
int64 pages_removed; /* heap pages removed by vacuum
@@ -192,10 +245,6 @@ typedef struct ExtVacReport
int64 pages_frozen; /* pages marked in VM as frozen */
int64 pages_all_visible; /* pages marked in VM as
* all-visible */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are
- * still visible to some
- * transaction */
int64 vm_new_frozen_pages; /* pages marked in VM as
* frozen */
int64 vm_new_visible_pages; /* pages marked in VM as
@@ -203,9 +252,6 @@ typedef struct ExtVacReport
int64 vm_new_visible_frozen_pages; /* pages marked in VM as
* all-visible and
* frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due
- * to failure to get a cleanup
- * lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
} table;
@@ -214,60 +260,21 @@ typedef struct ExtVacReport
int64 pages_deleted; /* number of pages deleted by vacuum */
} index;
} /* per_type_stats */ ;
-} ExtVacReport;
+} PgStat_VacuumRelationCounts;
-/* ----------
- * PgStat_TableCounts The actual per-table counts kept by a backend
- *
- * This struct should contain only actual event counters, because we make use
- * of pg_memory_is_all_zeros() to detect whether there are any stats updates
- * to apply.
- *
- * It is a component of PgStat_TableStatus (within-backend state).
- *
- * Note: for a table, tuples_returned is the number of tuples successfully
- * fetched by heap_getnext, while tuples_fetched is the number of tuples
- * successfully fetched by heap_fetch under the control of bitmap indexscans.
- * For an index, tuples_returned is the number of index entries returned by
- * the index AM, while tuples_fetched is the number of tuples successfully
- * fetched by heap_fetch under the control of simple indexscans for this index.
- *
- * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
- * actions, regardless of whether the transaction committed. delta_live_tuples,
- * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
- * Note that delta_live_tuples and delta_dead_tuples can be negative!
- * ----------
- */
-typedef struct PgStat_TableCounts
+typedef struct PgStat_VacuumRelationStatus
{
- PgStat_Counter numscans;
-
- PgStat_Counter tuples_returned;
- PgStat_Counter tuples_fetched;
-
- PgStat_Counter tuples_inserted;
- PgStat_Counter tuples_updated;
- PgStat_Counter tuples_deleted;
- PgStat_Counter tuples_hot_updated;
- PgStat_Counter tuples_newpage_updated;
- bool truncdropped;
-
- PgStat_Counter delta_live_tuples;
- PgStat_Counter delta_dead_tuples;
- PgStat_Counter changed_tuples;
-
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-
- PgStat_Counter rev_all_visible_pages;
- PgStat_Counter rev_all_frozen_pages;
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
- /*
- * Additional cumulative stat on vacuum operations. Use an expensive
- * structure as an abstraction for different types of relations.
- */
- ExtVacReport vacuum_ext;
-} PgStat_TableCounts;
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ PgStat_CommonCounts common;
+ int32 errors;
+} PgStat_VacuumDBCounts;
/* ----------
* PgStat_TableStatus Per-table status within a backend
@@ -293,6 +300,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -489,8 +502,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
-
- ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -578,8 +589,6 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter rev_all_visible_pages;
PgStat_Counter rev_all_frozen_pages;
-
- ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -788,7 +797,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport * params);
+ TimestampTz starttime);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -924,6 +933,16 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+ pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts * pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts * pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid);
+PgStat_VacuumDBCounts *pgstat_fetch_stat_vacuum_dbentry(Oid dbid);
+
/*
* Functions in pgstat_wal.c
*/
@@ -940,7 +959,8 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
-
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics_for_relations;
/*
* Variables in pgstat_bgwriter.c
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 7dffab8dbdd..4abe70cb54e 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -500,6 +500,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -678,6 +690,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index eb5f0b3ae6d..52e884fbf8b 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
diff --git a/src/test/recovery/t/050_vacuum_extending_basic_test.pl b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
index bd3cb544e30..e2fd541fd89 100644
--- a/src/test/recovery/t/050_vacuum_extending_basic_test.pl
+++ b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
@@ -28,6 +28,7 @@ $node->init;
# Configure the server logging level for the test
$node->append_conf('postgresql.conf', q{
log_min_messages = notice
+ track_vacuum_statistics = on
});
my $stderr;
@@ -64,8 +65,9 @@ $node->safe_psql($dbname, q{
$node->safe_psql(
$dbname,
- "CREATE TABLE vestat (x int PRIMARY KEY)
+ "CREATE TABLE vestat (x int)
WITH (autovacuum_enabled = off, fillfactor = 10);
+ create index vestat_pkey on vestat (x);
INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
ANALYZE vestat;"
);
@@ -115,7 +117,7 @@ sub wait_for_vacuum_stats {
AND
(SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
FROM pg_stat_vacuum_indexes
- WHERE relname = 'vestat_pkey');"
+ WHERE indexrelname = 'vestat_pkey');"
);
return 1 if ($result_query eq 't');
@@ -183,7 +185,7 @@ sub fetch_vacuum_stats {
$dbname,
"SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
FROM pg_stat_vacuum_indexes
- WHERE relname = 'vestat_pkey';"
+ WHERE indexrelname = 'vestat_pkey';"
);
$index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
@@ -321,7 +323,7 @@ sub fetch_error_base_idx_vacuum_statistics {
$dbname,
"SELECT tuples_deleted, pages_deleted
FROM pg_stat_vacuum_indexes
- WHERE relname = 'vestat_pkey';"
+ WHERE indexrelname = 'vestat_pkey';"
);
$base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
my ($cur_tuples_deleted, $cur_pages_deleted) = split /\s+/, $base_statistics;
@@ -343,7 +345,7 @@ sub fetch_error_wal_idx_vacuum_statistics {
$dbname,
"SELECT wal_records, wal_bytes, wal_fpi
FROM pg_stat_vacuum_indexes
- WHERE relname = 'vestat_pkey';"
+ WHERE indexrelname = 'vestat_pkey';"
);
$wal_raw =~ s/\s*\|\s*/ /g; # transform " | " in space
@@ -707,7 +709,7 @@ $base_stats = $node->safe_psql(
'postgres',
"SELECT count(*) = 0
FROM pg_stat_vacuum_indexes
- WHERE relname = 'vestat_pkey';"
+ WHERE indexrelname = 'vestat_pkey';"
);
ok($base_stats eq 't', 'check the printing index vacuum extended statistics from another database are not available');
@@ -742,6 +744,9 @@ $reloid = $node->safe_psql(
}
);
+# Run VACUUM on shared table to ensure stats entry is created
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
# Check if we can get vacuum statistics for cluster relations (dbid = 0)
$base_stats = $node->safe_psql(
$dbname,
@@ -760,6 +765,10 @@ my $indoid = $node->safe_psql(
}
);
+# Run VACUUM on shared index to ensure stats entry is created
+# Note: VACUUM on the table will also vacuum its indexes
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
$base_stats = $node->safe_psql(
$dbname,
qq{
diff --git a/src/test/recovery/t/051_vacuum_extending_freeze_test.pl b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
index 7528f20098b..2a1c506a22f 100644
--- a/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
+++ b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
@@ -37,6 +37,7 @@ $node->append_conf('postgresql.conf', q{
vacuum_max_eager_freeze_failure_rate = 1.0
vacuum_failsafe_age = 0
vacuum_multixact_failsafe_age = 0
+ track_vacuum_statistics = on
});
$node->start();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index b627c85e332..4e8b8b8a2b1 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2330,77 +2330,81 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
-pg_stat_vacuum_database| SELECT db.oid AS dboid,
- db.datname AS dbname,
- stats.db_blks_read,
- stats.db_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time,
- stats.wraparound_failsafe,
- stats.errors
- FROM pg_database db,
- LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
-pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
- ns.nspname AS schemaname,
- rel.relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_deleted,
- stats.tuples_deleted,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time
- FROM (pg_class rel
- JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
- LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
- WHERE (rel.relkind = 'i'::"char");
-pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
- rel.relname,
- stats.relid,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_scanned,
- stats.pages_removed,
- stats.vm_new_frozen_pages,
- stats.vm_new_visible_pages,
- stats.vm_new_visible_frozen_pages,
- stats.missed_dead_pages,
- stats.tuples_deleted,
- stats.tuples_frozen,
- stats.recently_dead_tuples,
- stats.missed_dead_tuples,
- stats.wraparound_failsafe,
- stats.index_vacuum_count,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time
- FROM (pg_class rel
- JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
- LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
- WHERE (rel.relkind = 'r'::"char");
+pg_stat_vacuum_database| SELECT d.oid AS dboid,
+ d.datname AS dbname,
+ s.db_blks_read,
+ s.db_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time,
+ s.wraparound_failsafe,
+ s.errors
+ FROM pg_database d,
+ LATERAL pg_stat_get_vacuum_database(d.oid) s(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
+pg_stat_vacuum_indexes| SELECT c.oid AS relid,
+ i.oid AS indexrelid,
+ n.nspname AS schemaname,
+ c.relname,
+ i.relname AS indexrelname,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_deleted,
+ s.tuples_deleted,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (((pg_class c
+ JOIN pg_index x ON ((c.oid = x.indrelid)))
+ JOIN pg_class i ON ((i.oid = x.indexrelid)))
+ LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(i.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
+pg_stat_vacuum_tables| SELECT n.nspname AS schemaname,
+ c.relname,
+ s.relid,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_scanned,
+ s.pages_removed,
+ s.vm_new_frozen_pages,
+ s.vm_new_visible_pages,
+ s.vm_new_visible_frozen_pages,
+ s.missed_dead_pages,
+ s.tuples_deleted,
+ s.tuples_frozen,
+ s.recently_dead_tuples,
+ s.missed_dead_tuples,
+ s.wraparound_failsafe,
+ s.index_vacuum_count,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (pg_class c
+ JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(c.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
--
2.39.5 (Apple Git-154)
From cfbf50c85ae9bfdfb9167dab446bf3256e33ddb1 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 5/5] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 755 +++++++++++++++++++++++++++++++++
1 file changed, 755 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 162c76b729a..50578cae90d 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5654,4 +5654,759 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v26-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (79.0K, 3-v26-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From f96d5079774fe129fff32761bba4ab9089e491bd Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 9 Dec 2025 09:56:34 +0300
Subject: [PATCH 1/5] Machinery for grabbing an extended vacuum statistics on
table relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen is a number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible is a number of pages that are marked as all-visible in vm during
vacuum.
wraparound_failsafe_count is a number of times when the vacuum starts urgent cleanup
to prevent wraparound problem which is critical for the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 145 ++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 52 +-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 46 +-
src/backend/utils/adt/pgstatfuncs.c | 86 +++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 18 +
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 92 ++-
.../vacuum-extending-in-repetable-read.out | 53 ++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 53 ++
.../t/050_vacuum_extending_basic_test.pl | 571 ++++++++++++++++++
.../t/051_vacuum_extending_freeze_test.pl | 395 ++++++++++++
src/test/regress/expected/rules.out | 44 +-
src/test/regress/parallel_schedule | 2 +-
18 files changed, 1565 insertions(+), 10 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/recovery/t/050_vacuum_extending_basic_test.pl
create mode 100644 src/test/recovery/t/051_vacuum_extending_freeze_test.pl
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 30778a15639..66e09d0a0cf 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -289,6 +289,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -407,6 +408,10 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to
+ * prevent anti-wraparound
+ * shutdown */
} LVRelState;
@@ -418,6 +423,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -487,6 +504,102 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ TimestampTz starttime;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ ExtVacReport * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
@@ -645,6 +758,13 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
char **indnames = NULL;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -673,6 +793,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
pgstat_progress_update_param(PROGRESS_VACUUM_STARTED_BY,
PROGRESS_VACUUM_STARTED_BY_MANUAL);
+ extvac_stats_start(rel, &extVacCounters);
+
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -689,6 +811,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -797,6 +920,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->aggressive = vacuum_get_cutoffs(rel, params, &vacrel->cutoffs);
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ vacrel->wraparound_failsafe_count = 0;
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
@@ -951,6 +1075,23 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -965,7 +1106,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime);
+ starttime,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -3019,6 +3161,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index d14588e92ae..3030242d98e 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -161,6 +162,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 0a0f95f6bb9..ffb407d414f 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -727,7 +727,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1452,3 +1454,51 @@ REVOKE ALL ON pg_aios FROM PUBLIC;
GRANT SELECT ON pg_aios TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_aios() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.vm_new_frozen_pages AS vm_new_frozen_pages,
+ stats.vm_new_visible_pages AS vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ stats.missed_dead_pages AS missed_dead_pages,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.recently_dead_tuples AS recently_dead_tuples,
+ stats.missed_dead_tuples AS missed_dead_tuples,
+
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 0528d1b6ecb..dd519447387 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -117,6 +117,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2536,6 +2539,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 8a37c08871a..114cd7c31d3 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1054,6 +1054,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 55a10c299db..361713479e8 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
+ bool accumulate_reltype_specific_info);
/*
@@ -208,7 +210,7 @@ pgstat_drop_relation(Relation rel)
*/
void
pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
- PgStat_Counter deadtuples, TimestampTz starttime)
+ PgStat_Counter deadtuples, TimestampTz starttime, ExtVacReport * params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -234,6 +236,8 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -880,6 +884,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -1009,3 +1016,40 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
+ dst->vm_new_visible_pages += src->vm_new_visible_pages;
+ dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->missed_dead_pages += src->missed_dead_pages;
+ dst->missed_dead_tuples += src->missed_dead_tuples;
+
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index ef6fffe60b9..d7dfda0c1a7 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2307,3 +2313,83 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index dc9e2255f8a..867638fe74b 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -669,6 +669,7 @@
#track_wal_io_timing = off
#track_functions = none # none, pl, all
#stats_fetch_consistency = cache # cache, none, snapshot
+#track_vacuum_statistics = off
# - Monitoring -
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fd9448ec7b9..915a5a7822f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12612,4 +12612,22 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int4,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 1f3290c7fbf..6b997bc7fb1 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -332,6 +332,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 6714363144a..46d12fa3bd0 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -114,6 +114,66 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /*
+ * number of blocks missed, hit, dirtied and written during a vacuum of
+ * specific relation
+ */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /*
+ * blocks missed and hit for just the heap during a vacuum of specific
+ * relation
+ */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images
+ * produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay
+ * point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as
+ * all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to
+ * failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still
+ * visible to some transaction */
+ int64 index_vacuum_count; /* the number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to
+ * prevent anti-wraparound
+ * shutdown */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -156,6 +216,15 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations. Use an expensive
+ * structure as an abstraction for different types of relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -214,7 +283,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -378,6 +447,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -461,8 +532,12 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autovacuum_time;
PgStat_Counter total_analyze_time;
PgStat_Counter total_autoanalyze_time;
-
TimestampTz stat_reset_time;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -671,7 +746,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
- TimestampTz starttime);
+ TimestampTz starttime, ExtVacReport * params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -722,6 +797,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..87f7e40b4a6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index f2e067b1fbc..1c231418706 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -98,6 +98,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5893d89573d
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,53 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/recovery/t/050_vacuum_extending_basic_test.pl b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..7e25a3fe63f
--- /dev/null
+++ b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
@@ -0,0 +1,571 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across tables using:
+#
+# • pg_stat_vacuum_tables
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server logging level for the test
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls pg_stat_vacuum_tables until the table-level counters exceed
+# the provided baselines, or until the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = $args{tab_tuples_deleted} or 0;
+ my $tab_wal_records = $args{tab_wal_records} or 0;
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $pages_frozen = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $pages_frozen_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table,
+# storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($pages_frozen, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $pages_frozen_prev = $pages_frozen;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " pages_frozen = $pages_frozen_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " pages_frozen = $pages_frozen\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats during mismatch
+#
+# Print current values and old values of relevant vacuum counters for the test
+# table, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_error_base_tab_vacuum_statistics {
+
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_frozen_pages, tuples_deleted, pages_scanned, pages_removed
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($cur_pages_frozen, $cur_tuples_deleted, $cur_pages_scanned, $cur_pages_removed) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR TABLE:\n" .
+ " Baseline:\n" .
+ " pages_frozen = $pages_frozen\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " Current:\n" .
+ " pages_frozen = $cur_pages_frozen\n" .
+ " tuples_deleted = $cur_tuples_deleted\n" .
+ " pages_scanned = $cur_pages_scanned\n" .
+ " pages_removed = $cur_pages_removed\n"
+ );
+}
+
+sub fetch_error_wal_tab_vacuum_statistics {
+
+ my $wal_raw = $node->safe_psql(
+ $dbname,
+ "SELECT wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $wal_raw =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($cur_wal_rec, $cur_wal_bytes, $cur_wal_fpi) = split /\s+/, $wal_raw;
+
+ diag(
+ "WAL STATS MISMATCH FOR TABLE:\n" .
+ " Baseline:\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " Current:\n" .
+ " wal_records = $cur_wal_rec\n" .
+ " wal_bytes = $cur_wal_bytes\n" .
+ " wal_fpi = $cur_wal_fpi\n"
+ );
+}
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM, and wait for stats to advance' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds after vacuuming half-deleted table";
+
+#------------------------------------------------------------------------------
+# Check statistics after half-table delete
+#------------------------------------------------------------------------------
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM, and wait for stats to advance' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds after vacuuming all-deleted table";
+
+#------------------------------------------------------------------------------
+# Check statistics after full delete
+#------------------------------------------------------------------------------
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted,
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Verify statistics after updating tuples and vacuuming
+#------------------------------------------------------------------------------
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => $wal_records_prev
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Verify statistics after updating full table, vacuum and trancation
+#------------------------------------------------------------------------------
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => $wal_records
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Verify statistics after table vacuum and trancation
+#------------------------------------------------------------------------------
+
+# Get current statistics
+fetch_vacuum_stats();
+
+ok($pages_frozen == $pages_frozen_prev, 'table pages_frozen stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error(); # End of subtest
+
+# Save statistics for the next test
+save_vacuum_stats();
+
+#-------------------------------------------------------------------------------------------------------
+# Test 8: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap elation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) = 1 FROM pg_stat_get_vacuum_tables($reloid);"
+);
+ok($base_stats eq 't', 'heap vacuum stats return from the current relation and database as expected');
+
+#------------------------------------------------------------------------------
+# Test 9: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 0
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+ok($base_stats eq 't', 'check the printing heap vacuum extended statistics from another database are not available');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+# Check if we can get vacuum statistics for cluster relations (dbid = 0)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) = 1
+ FROM pg_stat_get_vacuum_tables($reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+#------------------------------------------------------------------------------
+# Test 11: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*) = 0
+ FROM pg_stat_vacuum_tables WHERE relid = 0;
+ }
+);
+ok($base_stats eq 't', 'pg_stat_vacuum_tables correctly returns no rows for OID = 0');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;"
+);
+
+$node->stop;
+
+done_testing();
diff --git a/src/test/recovery/t/051_vacuum_extending_freeze_test.pl b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..a9b5d6cb739
--- /dev/null
+++ b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
@@ -0,0 +1,395 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats system using TAP
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server for aggressive freezing behavior used by the test
+# These settings ensure that VACUUM always freezes pages aggressively:
+# - vacuum_freeze_min_age = 0: freeze tuples as soon as possible (no age requirement)
+# - vacuum_freeze_table_age = 0: always perform aggressive scan (scan all pages)
+# - vacuum_multixact_freeze_min_age = 0: freeze multixacts as soon as possible
+# - vacuum_multixact_freeze_table_age = 0: always perform aggressive scan for multixacts
+# - vacuum_max_eager_freeze_failure_rate = 1.0: enable aggressive eager scanning (100% of pages)
+# - vacuum_failsafe_age = 0: disable failsafe (for testing)
+# - vacuum_multixact_failsafe_age = 0: disable multixact failsafe (for testing)
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Enable necessary settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+# wait_for_vacuum_stats
+#
+# Polls pg_stat_vacuum_tables until the named columns exceed the provided
+# baseline values or until timeout. Callers should pass:
+#
+# tab_frozen_column => 'vm_new_frozen_pages' # column name (string) or 'rev_all_frozen_pages'
+# tab_visible_column => 'vm_new_visible_pages' # column name (string) or 'rev_all_visible_pages'
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 or 1 # if true, run vacuum_sql before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_frozen_column = $args{tab_frozen_column};
+ my $tab_visible_column = $args{tab_visible_column};
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count};
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count};
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ $sql = "
+ SELECT ($tab_frozen_column > $tab_all_frozen_pages_count AND
+ $tab_visible_column > $tab_all_visible_pages_count)
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_rev_all_frozen_pages(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_rev_all_visible_pages(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ # sub-second sleep
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_frozen_pages;
+my $vm_new_visible_pages;
+
+my $rev_all_frozen_pages;
+my $rev_all_visible_pages;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ $vm_new_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_frozen_pages
+ FROM pg_stat_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $vm_new_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_pages
+ FROM pg_stat_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats during mismatch
+#
+# Print current values and old values of relevant vacuum counters for the test
+# table, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_error_tab_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "tab_column required"
+ unless exists $args{tab_column} && defined $args{tab_column};
+ die "tab_value required"
+ unless exists $args{tab_value};
+
+ my $tab_column = $args{tab_column};
+ my $tab_value = $args{tab_value};
+
+ # fetch actual base vacuum statistics
+ my $cur_value = $node->safe_psql(
+ $dbname,
+ "SELECT $tab_column
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ diag("MISMATCH FOR $tab_column: the current value is $cur_value, while it should be $tab_value");
+}
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+$node->safe_psql($dbname, q{
+ SELECT pg_stat_force_next_flush();
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+# Poll the stats view until the expected deltas appear or timeout.
+# We do not expect rev_all_* counters to change here, so we pass -1 for them.
+$updated = wait_for_vacuum_stats(
+ tab_frozen_column => 'vm_new_frozen_pages',
+ tab_visible_column => 'vm_new_visible_pages',
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_frozen_pages and vm_new_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Verify initial statistics after vacuum
+#------------------------------------------------------------------------------
+
+$res = $node->safe_psql($dbname, q{
+ SELECT vm_new_frozen_pages > 0 FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+});
+ok($res eq 't', 'vacuum froze some pages, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_frozen_pages', tab_value => $vm_new_frozen_pages,);
+
+$res = $node->safe_psql($dbname, q{
+ SELECT vm_new_visible_pages > 0 FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+});
+ok($res eq 't', 'vacuum marked pages all-visible, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_visible_pages', tab_value =>$vm_new_visible_pages,);
+
+$res = $node->safe_psql($dbname, q{
+ SELECT pg_stat_get_rev_all_frozen_pages(c.oid) = 0
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';
+});
+ok($res eq 't', 'vacuum did not increase frozen-page revision count, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_frozen_pages', tab_value => 0,);
+
+$res = $node->safe_psql($dbname, q{
+ SELECT pg_stat_get_rev_all_visible_pages(c.oid) = 0
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';
+});
+ok($res eq 't', 'vacuum did not increase visible-page revision count, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_visible_pages', tab_value => 0,);
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+# Poll until stats update or timeout.
+# We do not expect vm_new_frozen_pages or vm_new_visible_pages to change here,
+# so we pass -1 for those counters.
+$updated = wait_for_vacuum_stats(
+ tab_frozen_column => 'rev_all_frozen_pages',
+ tab_visible_column => 'rev_all_visible_pages',
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Check updated statistics after backend activity
+#------------------------------------------------------------------------------
+
+$res = $node->safe_psql($dbname,
+ "SELECT vm_new_frozen_pages = $vm_new_frozen_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+);
+ok($res eq 't', 'backend activity did not increase the frozen-page count') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_frozen_pages', tab_value => $vm_new_frozen_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT vm_new_visible_pages = $vm_new_visible_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+);
+ok($res eq 't', 'backend activity did not increase the all-visible page count') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_visible_pages', tab_value => $vm_new_visible_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid) > $rev_all_frozen_pages
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';"
+);
+ok($res eq 't', 'backend activity increased frozen-page revision count') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_frozen_pages', tab_value => $rev_all_frozen_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid) > $rev_all_visible_pages
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';"
+);
+ok($res eq 't', 'backend activity increased visible-page revision count') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_visible_pages', tab_value => $rev_all_visible_pages,);
+
+#------------------------------------------------------------------------------
+# Update saved snapshots
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+
+$node->safe_psql($dbname, q{ VACUUM (FREEZE, VERBOSE) vestat; });
+
+# Poll until stats update or timeout.
+# We pass current snapshot values for vm_new_frozen_pages/vm_new_visible_pages and expect rev counters unchanged.
+$updated = wait_for_vacuum_stats(
+ tab_frozen_column => 'vm_new_frozen_pages',
+ tab_visible_column => 'vm_new_visible_pages',
+ tab_all_frozen_pages_count => $vm_new_frozen_pages,
+ tab_all_visible_pages_count => $vm_new_visible_pages,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_frozen_pages and vm_new_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Verify statistics after final vacuum
+# Check updated stats after backend work
+#------------------------------------------------------------------------------
+$res = $node->safe_psql($dbname,
+ "SELECT vm_new_frozen_pages > $vm_new_frozen_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+);
+ok($res eq 't', 'vacuum froze some pages after backend activity, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_frozen_pages', tab_value => $vm_new_frozen_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT vm_new_visible_pages > $vm_new_visible_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+);
+ok($res eq 't', 'vacuum marked pages all-visible after backend activity, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_visible_pages', tab_value => $vm_new_visible_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid) = $rev_all_frozen_pages
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';"
+);
+ok($res eq 't', 'vacuum did not increase frozen-page revision count after backend activity, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_frozen_pages', tab_value => $rev_all_frozen_pages,);
+
+$res = $node->safe_psql($dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid) = $rev_all_visible_pages
+ FROM pg_stat_vacuum_tables vt
+ JOIN pg_class c ON c.relname = vt.relname
+ WHERE vt.relname = 'vestat';"
+);
+ok($res eq 't', 'vacuum did not increase visible-page revision count after backend activity, as expected') or
+ fetch_error_tab_vacuum_statistics(tab_column => 'rev_all_visible_pages', tab_value => $rev_all_visible_pages,);
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 4286c266e17..e4a77878beb 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1844,7 +1844,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2266,7 +2268,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2321,9 +2325,43 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.missed_dead_pages,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.missed_dead_tuples,
+ stats.wraparound_failsafe,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 905f9bca959..62f2ac11659 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -139,4 +139,4 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
-test: tablespace
+test: tablespace
\ No newline at end of file
--
2.39.5 (Apple Git-154)
[text/plain] v26-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (54.6K, 4-v26-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From d09c2f688fc8776b239a39f0cd9cda5488dba812 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 9 Dec 2025 10:56:54 +0300
Subject: [PATCH 2/5] Machinery for grabbing an extended vacuum statistics on
index relations.
They are gathered separatelly from table statistics.
As for tables, we gather vacuum shared buffers statistics for index relations like
value of total_blks_hit, total_blks_read, total_blks_dirtied, wal statistics, io time
during flushing buffer pages to disk, delay and total time.
Due to the fact that such statistics are common as for tables, as for indexes we
set them in the union ExtVacReport structure. We only added some determination 'type'
field to highlight what kind belong to these statistics: PGSTAT_EXTVAC_TABLE or
PGSTAT_EXTVAC_INDEX. Generally, PGSTAT_EXTVAC_INVALID type leads to wrong code process.
Some statistics belong only one type of both tables or indexes. So, we added substructures
sych table and index inside ExtVacReport structure.
Therefore, we gather only for tables such statistics like number of scanned, removed pages,
their charecteristics according VM (all-visible and frozen). In addition, for tables we
gather number frozen, deleted and recently dead tuples and how many times vacuum processed
indexes for tables.
Controversally for indexes we gather number of deleted pages and deleted tuples only.
As for tables, deleted pages and deleted tuples reflect the overall performance of the vacuum
for the index relationship.
Since the vacuum cleans up references to tuple indexes before cleaning up table tuples,
which adds some complexity to the vacuum process, namely the vacuum switches from cleaning up
a table to its indexes and back during its operation, we need to save the vacuum statistics
collected for the heap before it starts cleaning up the indexes.
That's why it's necessary to track the vacuum statistics for the heap several times during
the vacuum procedure. To avoid sending the statistics to the Cumulative Statistics System
several times, we save these statistics in the LVRelState structure and only after vacuum
finishes cleaning up the heap, it sends them to the Cumulative Statistics System.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 232 +++++++++++++----
src/backend/catalog/system_views.sql | 32 +++
src/backend/commands/vacuumparallel.c | 10 +
src/backend/utils/activity/pgstat_relation.c | 45 ++--
src/backend/utils/adt/pgstatfuncs.c | 92 ++++++-
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 25 ++
src/include/pgstat.h | 77 ++++--
.../vacuum-extending-in-repetable-read.out | 4 +-
.../t/050_vacuum_extending_basic_test.pl | 237 +++++++++++++++++-
src/test/regress/expected/rules.out | 22 ++
11 files changed, 681 insertions(+), 104 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 66e09d0a0cf..719ce90d96d 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -290,6 +290,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -412,6 +413,7 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to
* prevent anti-wraparound
* shutdown */
+ ExtVacReport extVacReportIdx;
} LVRelState;
@@ -423,19 +425,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-/*
- * Counters and usage data for extended stats tracking.
- */
-typedef struct LVExtStatCounters
-{
- TimestampTz starttime;
- WalUsage walusage;
- BufferUsage bufusage;
- double VacuumDelayTime;
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-} LVExtStatCounters;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static void heap_vacuum_eager_scan_setup(LVRelState *vacrel,
@@ -565,27 +554,25 @@ extvac_stats_end(Relation rel, LVExtStatCounters * counters,
endtime = GetCurrentTimestamp();
TimestampDifference(counters->starttime, endtime, &secs, &usecs);
- memset(report, 0, sizeof(ExtVacReport));
-
/*
* Fill additional statistics on a vacuum processing operation.
*/
- report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
- report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
- report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
- report->total_blks_written = bufusage.shared_blks_written;
+ report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written += bufusage.shared_blks_written;
- report->wal_records = walusage.wal_records;
- report->wal_fpi = walusage.wal_fpi;
- report->wal_bytes = walusage.wal_bytes;
+ report->wal_records += walusage.wal_records;
+ report->wal_fpi += walusage.wal_fpi;
+ report->wal_bytes += walusage.wal_bytes;
- report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
- report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
- report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
- report->total_time = secs * 1000. + usecs / 1000.;
+ report->total_time += secs * 1000. + usecs / 1000.;
if (!rel->pgstat_info || !pgstat_track_counts)
@@ -595,12 +582,122 @@ extvac_stats_end(Relation rel, LVExtStatCounters * counters,
*/
return;
- report->blks_fetched =
+ report->blks_fetched +=
rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
- report->blks_hit =
+ report->blks_hit +=
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ /* Set initial values for common heap and index statistics */
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, ExtVacReport * report)
+{
+ memset(report, 0, sizeof(ExtVacReport));
+
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacStats)
+{
+ /* Fill heap-specific extended stats fields */
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
+ extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
+ extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
+ extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
+ extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
+ extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
+ extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
+ extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
+ extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+
+ extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
+ extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+
+}
+
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacIdxStats)
+{
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
+ vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
+ vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
+ vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
+ vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
+ vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
+ vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
+ vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
+ vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
+ vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
+
+ vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -760,11 +857,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
char **indnames = NULL;
LVExtStatCounters extVacCounters;
ExtVacReport extVacReport;
- ExtVacReport allzero;
/* Initialize vacuum statistics */
- memset(&allzero, 0, sizeof(ExtVacReport));
- extVacReport = allzero;
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -820,6 +915,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
vac_open_indexes(vacrel->rel, RowExclusiveLock, &vacrel->nindexes,
@@ -1078,20 +1175,6 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Make generic extended vacuum stats report */
extvac_stats_end(rel, &extVacCounters, &extVacReport);
- /* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
- extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
- extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
- extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
- extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
-
/*
* Report results to the cumulative stats system, too.
*
@@ -1102,6 +1185,13 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
+
+ /*
+ * Make generic extended vacuum stats report and fill heap-specific
+ * extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
pgstat_report_vacuum(rel,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
@@ -2811,10 +2901,20 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
* that parallel VACUUM only gets the precheck and this postcheck.
@@ -3244,10 +3344,20 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ ExtVacReport extVacReport;
+
+ memset(&extVacReport, 0, sizeof(ExtVacReport));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
estimated_count);
+
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
/* Reset the progress counters */
@@ -3273,6 +3383,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3291,6 +3406,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3299,6 +3415,15 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(indrel,
+ 0, 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3323,6 +3448,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3342,12 +3472,22 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum(indrel,
+ 0, 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ffb407d414f..47b6a00d297 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1502,3 +1502,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 114cd7c31d3..43450685b09 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,11 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum(indrel,
+ 0, 0, 0, &extVacReport);
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 361713479e8..4bd6afc3794 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -1036,20 +1036,35 @@ pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
- dst->vm_new_visible_pages += src->vm_new_visible_pages;
- dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->missed_dead_pages += src->missed_dead_pages;
- dst->missed_dead_tuples += src->missed_dead_tuples;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->tuples_deleted += src->tuples_deleted;
+ }
+ }
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index d7dfda0c1a7..755751c3b46 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2360,18 +2360,19 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2393,3 +2394,72 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 915a5a7822f..e957781b623 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12630,4 +12630,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 6b997bc7fb1..b48ace6084b 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -300,6 +301,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -413,4 +434,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, ExtVacReport * report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 46d12fa3bd0..f2881dbb6f9 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -114,11 +114,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -155,23 +163,58 @@ typedef struct ExtVacReport
* point, in msec */
double total_time; /* total time of a vacuum operation, in msec */
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
- int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
- int64 vm_new_visible_frozen_pages; /* pages marked in VM as
- * all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to
- * failure to get a cleanup lock */
- int64 missed_dead_pages; /* pages with missed dead tuples */
int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still
- * visible to some transaction */
- int64 index_vacuum_count; /* the number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency vacuums to
- * prevent anti-wraparound
- * shutdown */
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by
+ * VM) */
+ int64 pages_removed; /* heap pages removed by vacuum
+ * "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as
+ * all-visible */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are
+ * still visible to some
+ * transaction */
+ int64 vm_new_frozen_pages; /* pages marked in VM as
+ * frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as
+ * all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as
+ * all-visible and
+ * frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due
+ * to failure to get a cleanup
+ * lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ int32 wraparound_failsafe_count; /* number of emergency
+ * vacuums to prevent
+ * anti-wraparound
+ * shutdown */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */ ;
} ExtVacReport;
/* ----------
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 87f7e40b4a6..6d960423912 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -34,7 +34,7 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
(1 row)
step s1_commit: COMMIT;
@@ -48,6 +48,6 @@ step s2_print_vacuum_stats_table:
relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
--------------------------+--------------+--------------------+------------------+-----------------+-------------
-test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
(1 row)
diff --git a/src/test/recovery/t/050_vacuum_extending_basic_test.pl b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
index 7e25a3fe63f..8f7b1e2909b 100644
--- a/src/test/recovery/t/050_vacuum_extending_basic_test.pl
+++ b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
@@ -2,9 +2,10 @@
# Test cumulative vacuum stats system using TAP
#
# This test validates the accuracy and behavior of cumulative vacuum statistics
-# across tables using:
+# across heap tables, indexes using:
#
# • pg_stat_vacuum_tables
+# • pg_stat_vacuum_indexes
#
# A polling helper function repeatedly checks the stats views until expected
# deltas appear or a configurable timeout expires. This guarantees that
@@ -62,7 +63,7 @@ $node->safe_psql($dbname, q{
$node->safe_psql(
$dbname,
- "CREATE TABLE vestat (x int)
+ "CREATE TABLE vestat (x int PRIMARY KEY)
WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
ANALYZE vestat;"
@@ -80,12 +81,15 @@ my $updated = 0;
#------------------------------------------------------------------------------
# wait_for_vacuum_stats
#
-# Polls pg_stat_vacuum_tables until the table-level counters exceed
-# the provided baselines, or until the configured timeout elapses.
+# Polls pg_stat_vacuum_tables and pg_stat_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
#
# Expected named args (baseline values):
# tab_tuples_deleted
# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
#
# Returns: 1 if the condition is met before timeout, 0 otherwise.
#------------------------------------------------------------------------------
@@ -94,6 +98,8 @@ sub wait_for_vacuum_stats {
my (%args) = @_;
my $tab_tuples_deleted = $args{tab_tuples_deleted} or 0;
my $tab_wal_records = $args{tab_wal_records} or 0;
+ my $idx_tuples_deleted = $args{idx_tuples_deleted} or 0;
+ my $idx_wal_records = $args{idx_wal_records} or 0;
my $start = time();
while ((time() - $start) < $timeout) {
@@ -101,9 +107,14 @@ sub wait_for_vacuum_stats {
my $result_query = $node->safe_psql(
$dbname,
"VACUUM vestat;
- SELECT tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
FROM pg_stat_vacuum_tables
- WHERE relname = 'vestat';"
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM pg_stat_vacuum_indexes
+ WHERE relname = 'vestat_pkey');"
);
return 1 if ($result_query eq 't');
@@ -126,6 +137,12 @@ my $wal_records = 0;
my $wal_bytes = 0;
my $wal_fpi = 0;
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
my $pages_frozen_prev = 0;
my $tuples_deleted_prev = 0;
my $pages_scanned_prev = 0;
@@ -134,11 +151,17 @@ my $wal_records_prev = 0;
my $wal_bytes_prev = 0;
my $wal_fpi_prev = 0;
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
#------------------------------------------------------------------------------
# fetch_vacuum_stats
#
-# Reads current values of relevant vacuum counters for the test table,
-# storing them in package variables for subsequent comparisons.
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
#------------------------------------------------------------------------------
sub fetch_vacuum_stats {
@@ -153,6 +176,18 @@ sub fetch_vacuum_stats {
$base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
($pages_frozen, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
= split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_indexes
+ WHERE relname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
}
#------------------------------------------------------------------------------
@@ -169,6 +204,12 @@ sub save_vacuum_stats {
$wal_records_prev = $wal_records;
$wal_bytes_prev = $wal_bytes;
$wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
}
#------------------------------------------------------------------------------
@@ -195,7 +236,20 @@ sub print_vacuum_stats_on_error {
" pages_removed = $pages_removed\n" .
" wal_records = $wal_records\n" .
" wal_bytes = $wal_bytes\n" .
- " wal_fpi = $wal_fpi\n"
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_removed = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_removed = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
);
};
@@ -203,7 +257,8 @@ sub print_vacuum_stats_on_error {
# fetch_vacuum_stats during mismatch
#
# Print current values and old values of relevant vacuum counters for the test
-# table, storing them in package variables for subsequent comparisons.
+# table and its primary index, storing them in package variables for subsequent
+# comparisons.
#------------------------------------------------------------------------------
sub fetch_error_base_tab_vacuum_statistics {
@@ -258,6 +313,54 @@ sub fetch_error_wal_tab_vacuum_statistics {
);
}
+sub fetch_error_base_idx_vacuum_statistics {
+
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted
+ FROM pg_stat_vacuum_indexes
+ WHERE relname = 'vestat_pkey';"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($cur_tuples_deleted, $cur_pages_deleted) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR INDEX:\n" .
+ " Baseline:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_removed = $index_pages_deleted\n" .
+ " Current:\n" .
+ " tuples_deleted = $cur_tuples_deleted\n" .
+ " pages_deleted = $cur_pages_deleted\n"
+ );
+}
+
+sub fetch_error_wal_idx_vacuum_statistics {
+
+ my $wal_raw = $node->safe_psql(
+ $dbname,
+ "SELECT wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_indexes
+ WHERE relname = 'vestat_pkey';"
+ );
+
+ $wal_raw =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($cur_wal_rec, $cur_wal_bytes, $cur_wal_fpi) = split /\s+/, $wal_raw;
+
+ diag(
+ "WAL STATS MISMATCH FOR INDEX:\n" .
+ " Baseline:\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n" .
+ " Current:\n" .
+ " wal_records = $cur_wal_rec\n" .
+ " wal_bytes = $cur_wal_bytes\n" .
+ " wal_fpi = $cur_wal_fpi\n"
+ );
+}
+
#------------------------------------------------------------------------------
# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
#------------------------------------------------------------------------------
@@ -270,7 +373,9 @@ $node->safe_psql($dbname, "VACUUM vestat;");
# Poll the stats view until expected deltas appear or timeout
$updated = wait_for_vacuum_stats(
tab_tuples_deleted => 0,
- tab_wal_records => 0
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
);
ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds after vacuuming half-deleted table";
@@ -290,6 +395,12 @@ ok($wal_records > $wal_records_prev, 'table wal_records has increased');
ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -307,6 +418,8 @@ $node->safe_psql($dbname, "VACUUM vestat;");
$updated = wait_for_vacuum_stats(
tab_tuples_deleted => $tuples_deleted_prev,
tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
);
ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
@@ -327,6 +440,12 @@ ok($wal_records > $wal_records_prev, 'table wal_records has increased');
ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -357,6 +476,12 @@ ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -379,6 +504,8 @@ $node->safe_psql(
$updated = wait_for_vacuum_stats(
tab_tuples_deleted => $tuples_deleted,
tab_wal_records => $wal_records,
+ idx_tuples_deleted => $index_tuples_deleted,
+ idx_wal_records => $index_wal_records,
);
ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
@@ -399,6 +526,12 @@ ok($wal_records > $wal_records_prev, 'table wal_records has increased');
ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -421,7 +554,9 @@ $node->safe_psql($dbname, "VACUUM vestat;");
$updated = wait_for_vacuum_stats(
tab_tuples_deleted => 0,
- tab_wal_records => $wal_records_prev
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
);
ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (tuples_deleted and wal_records advanced)')
@@ -442,6 +577,12 @@ ok($wal_records > $wal_records_prev, 'table wal_records has increased');
ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -464,7 +605,9 @@ $node->safe_psql(
$updated = wait_for_vacuum_stats(
tab_tuples_deleted => 0,
- tab_wal_records => $wal_records
+ tab_wal_records => $wal_records,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
);
ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (tuples_deleted and wal_records advanced)')
@@ -485,6 +628,12 @@ ok($wal_records > $wal_records_prev, 'table wal_records has increased');
ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
} or print_vacuum_stats_on_error(); # End of subtest
# Save statistics for the next test
@@ -513,6 +662,34 @@ $base_stats = $node->safe_psql(
);
ok($base_stats eq 't', 'heap vacuum stats return from the current relation and database as expected');
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) = 1 FROM pg_stat_vacuum_indexes($dboid, $reloid);"
+);
+ok($base_stats eq 't', 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) = 0 FROM pg_stats_vacuum_tables($dboid, 1);"
+);
+ok($base_stats eq 't', 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) = 0 FROM pg_stat_vacuum_indexes($dboid, 1);"
+);
+ok($base_stats eq 't', 'index vacuum stats return no rows, as expected');
+
+
#------------------------------------------------------------------------------
# Test 9: Check relation-level vacuum statistics from another database
#------------------------------------------------------------------------------
@@ -523,6 +700,14 @@ $base_stats = $node->safe_psql(
FROM pg_stat_vacuum_tables
WHERE relname = 'vestat';"
);
+ok($base_stats eq 't', 'check the printing table vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 0
+ FROM pg_stat_vacuum_indexes
+ WHERE relname = 'vestat_pkey';"
+);
ok($base_stats eq 't', 'check the printing heap vacuum extended statistics from another database are not available');
$reloid = $node->safe_psql(
@@ -543,6 +728,23 @@ $base_stats = $node->safe_psql(
is($base_stats, 't', 'vacuum stats for common heap objects available');
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) = 1
+ FROM pg_stat_vacuum_indexes(0, $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
#------------------------------------------------------------------------------
# Test 11: Cleanup checks: ensure functions return empty sets for OID = 0
#------------------------------------------------------------------------------
@@ -562,6 +764,15 @@ $base_stats = $node->safe_psql(
);
ok($base_stats eq 't', 'pg_stat_vacuum_tables correctly returns no rows for OID = 0');
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*) = 0
+ FROM pg_stat_vacuum_indexes WHERE relid = 0;
+ }
+);
+ok($base_stats eq 't', 'pg_stat_vacuum_indexes correctly returns no rows for OID = 0');
+
$node->safe_psql('postgres',
"DROP DATABASE $dbname;"
);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index e4a77878beb..7e6029394cb 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2330,6 +2330,28 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schemaname,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
rel.relname,
stats.relid,
--
2.39.5 (Apple Git-154)
[text/plain] v26-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (22.4K, 5-v26-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 2051db2600566dc88040cee97868469aa35440d7 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 1 Sep 2025 21:43:33 +0300
Subject: [PATCH 3/5] Machinery for grabbing an extended vacuum statistics on
databases.
Database vacuum statistics information is the collected general
vacuum statistics indexes and tables owned by the databases, which
they belong to.
In addition to the fact that there are far fewer databases in a system
than relations, vacuum statistics for a database contain fewer statistics
than relations, but they are enough to indicate that something may be
wrong in the system and prompt the administrator to enable extended
monitoring for relations.
So, buffer, wal, statistics of I/O time of read and writen blocks
statistics will be observed because they are collected for both
tables, indexes. In addition, we show the number of errors caught
during operation of the vacuum only for the error level.
wraparound_failsafe_count is a number of times when the vacuum starts
urgent cleanup to prevent wraparound problem which is critical for
the database.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 2 +-
src/backend/catalog/system_views.sql | 26 +++++++-
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 13 +++-
src/backend/utils/adt/pgstatfuncs.c | 62 ++++++++++++++++++-
src/include/catalog/pg_proc.dat | 13 +++-
src/include/pgstat.h | 7 +--
.../vacuum-extending-in-repetable-read.spec | 6 ++
.../t/050_vacuum_extending_basic_test.pl | 49 ++++++++++++---
.../t/051_vacuum_extending_freeze_test.pl | 48 +++++---------
src/test/regress/expected/rules.out | 17 +++++
11 files changed, 195 insertions(+), 49 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 719ce90d96d..fcd92a43dda 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -663,7 +663,7 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacStats
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 47b6a00d297..dc86b1ee212 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1533,4 +1533,28 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.wraparound_failsafe AS wraparound_failsafe
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index b31f20d41bc..65207d30378 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -485,6 +485,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 4bd6afc3794..2675c541369 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -215,6 +215,7 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -273,6 +274,16 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
}
/*
@@ -1032,6 +1043,7 @@ pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
dst->blk_write_time += src->blk_write_time;
dst->delay_time += src->delay_time;
dst->total_time += src->total_time;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
if (!accumulate_reltype_specific_info)
return;
@@ -1059,7 +1071,6 @@ pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
dst->table.index_vacuum_count += src->table.index_vacuum_count;
dst->table.missed_dead_pages += src->table.missed_dead_pages;
dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
}
else if (dst->type == PGSTAT_EXTVAC_INDEX)
{
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 755751c3b46..4e2714f2e6a 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2371,7 +2371,7 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
@@ -2463,3 +2463,63 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index e957781b623..c3a2adb96f1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12631,12 +12631,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,wraparound_failsafe}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f2881dbb6f9..f3bdc1c38df 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -165,6 +165,9 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
+ int32 wraparound_failsafe_count; /* the number of times to prevent
+ * wraparound problem */
+
ExtVacReportType type; /* heap, index, etc. */
/* ----------
@@ -205,10 +208,6 @@ typedef struct ExtVacReport
* lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
- int32 wraparound_failsafe_count; /* number of emergency
- * vacuums to prevent
- * anti-wraparound
- * shutdown */
} table;
struct
{
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5893d89573d..cfec3159580 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -18,6 +18,9 @@ teardown
}
session s1
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s1_begin_repeatable_read {
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
select count(ival) from test_vacuum_stat_isolation where id>900;
@@ -25,6 +28,9 @@ step s1_begin_repeatable_read {
step s1_commit { COMMIT; }
session s2
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
diff --git a/src/test/recovery/t/050_vacuum_extending_basic_test.pl b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
index 8f7b1e2909b..bd3cb544e30 100644
--- a/src/test/recovery/t/050_vacuum_extending_basic_test.pl
+++ b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
@@ -2,10 +2,11 @@
# Test cumulative vacuum stats system using TAP
#
# This test validates the accuracy and behavior of cumulative vacuum statistics
-# across heap tables, indexes using:
+# across heap tables, indexes, and databases using:
#
# • pg_stat_vacuum_tables
# • pg_stat_vacuum_indexes
+# • pg_stat_vacuum_database
#
# A polling helper function repeatedly checks the stats views until expected
# deltas appear or a configurable timeout expires. This guarantees that
@@ -672,20 +673,20 @@ $reloid = $node->safe_psql(
# Check if we can get vacuum statistics of particular index relation in the current database
$base_stats = $node->safe_psql(
$dbname,
- "SELECT count(*) = 1 FROM pg_stat_vacuum_indexes($dboid, $reloid);"
+ "SELECT count(*) = 1 FROM pg_stat_get_vacuum_indexes($reloid);"
);
ok($base_stats eq 't', 'index vacuum stats return from the current relation and database as expected');
# Check if we return empty results if vacuum statistics with particular oid doesn't exist
$base_stats = $node->safe_psql(
$dbname,
- "SELECT count(*) = 0 FROM pg_stats_vacuum_tables($dboid, 1);"
+ "SELECT count(*) = 0 FROM pg_stat_get_vacuum_tables(1);"
);
ok($base_stats eq 't', 'table vacuum stats return no rows, as expected');
$base_stats = $node->safe_psql(
$dbname,
- "SELECT count(*) = 0 FROM pg_stat_vacuum_indexes($dboid, 1);"
+ "SELECT count(*) = 0 FROM pg_stat_get_vacuum_indexes(1);"
);
ok($base_stats eq 't', 'index vacuum stats return no rows, as expected');
@@ -708,7 +709,31 @@ $base_stats = $node->safe_psql(
FROM pg_stat_vacuum_indexes
WHERE relname = 'vestat_pkey';"
);
-ok($base_stats eq 't', 'check the printing heap vacuum extended statistics from another database are not available');
+ok($base_stats eq 't', 'check the printing index vacuum extended statistics from another database are not available');
+
+#--------------------------------------------------------------------------------------
+# Test 10: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit > 0 AND total_blks_dirtied > 0
+ AND total_blks_written > 0 AND wal_records > 0
+ AND wal_fpi > 0 AND wal_bytes > 0
+ FROM pg_stat_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = pg_stat_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from the current database are available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) > 0
+ FROM pg_stat_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = pg_stat_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
$reloid = $node->safe_psql(
$dbname,
@@ -739,7 +764,7 @@ $base_stats = $node->safe_psql(
$dbname,
qq{
SELECT count(*) = 1
- FROM pg_stat_vacuum_indexes(0, $indoid);
+ FROM pg_stat_get_vacuum_indexes($indoid);
}
);
@@ -773,8 +798,18 @@ $base_stats = $node->safe_psql(
);
ok($base_stats eq 't', 'pg_stat_vacuum_indexes correctly returns no rows for OID = 0');
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT COUNT(*) = 0
+ FROM pg_stat_vacuum_database WHERE dboid = 0;
+ }
+);
+ok($base_stats eq 't', 'pg_stat_vacuum_database correctly returns no rows for OID = 0');
+
$node->safe_psql('postgres',
- "DROP DATABASE $dbname;"
+ "DROP DATABASE $dbname;
+ VACUUM;"
);
$node->stop;
diff --git a/src/test/recovery/t/051_vacuum_extending_freeze_test.pl b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
index a9b5d6cb739..7528f20098b 100644
--- a/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
+++ b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
@@ -91,11 +91,17 @@ sub wait_for_vacuum_stats {
my $start = time();
my $sql;
+ my $vacuum_run = 0;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ $vacuum_run = 1;
+ }
while ((time() - $start) < $timeout) {
if ($run_vacuum) {
- $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
$sql = "
SELECT ($tab_frozen_column > $tab_all_frozen_pages_count AND
$tab_visible_column > $tab_all_visible_pages_count)
@@ -213,20 +219,6 @@ $node->safe_psql($dbname, q{
VACUUM (FREEZE, VERBOSE) vestat;
});
-# Poll the stats view until the expected deltas appear or timeout.
-# We do not expect rev_all_* counters to change here, so we pass -1 for them.
-$updated = wait_for_vacuum_stats(
- tab_frozen_column => 'vm_new_frozen_pages',
- tab_visible_column => 'vm_new_visible_pages',
- tab_all_frozen_pages_count => 0,
- tab_all_visible_pages_count => 0,
- run_vacuum => 1,
-);
-
-ok($updated,
- 'vacuum stats updated after vacuuming the table (vm_new_frozen_pages and vm_new_visible_pages advanced)')
- or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
-
#------------------------------------------------------------------------------
# Snapshot current statistics for later comparison
#------------------------------------------------------------------------------
@@ -238,7 +230,7 @@ fetch_vacuum_stats();
#------------------------------------------------------------------------------
$res = $node->safe_psql($dbname, q{
- SELECT vm_new_frozen_pages > 0 FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ SELECT vm_new_frozen_pages = 0 FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
});
ok($res eq 't', 'vacuum froze some pages, as expected') or
fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_frozen_pages', tab_value => $vm_new_frozen_pages,);
@@ -335,32 +327,24 @@ fetch_vacuum_stats();
$node->safe_psql($dbname, q{ VACUUM (FREEZE, VERBOSE) vestat; });
-# Poll until stats update or timeout.
-# We pass current snapshot values for vm_new_frozen_pages/vm_new_visible_pages and expect rev counters unchanged.
-$updated = wait_for_vacuum_stats(
- tab_frozen_column => 'vm_new_frozen_pages',
- tab_visible_column => 'vm_new_visible_pages',
- tab_all_frozen_pages_count => $vm_new_frozen_pages,
- tab_all_visible_pages_count => $vm_new_visible_pages,
- run_vacuum => 1,
-);
-
-ok($updated,
- 'vacuum stats updated after vacuuming the all-updated table (vm_new_frozen_pages and vm_new_visible_pages advanced)')
- or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
-
#------------------------------------------------------------------------------
# Verify statistics after final vacuum
# Check updated stats after backend work
#------------------------------------------------------------------------------
+
+# Fetch updated statistics to get the new baseline for comparison
+my $old_vm_new_frozen_pages = $vm_new_frozen_pages;
+my $old_vm_new_visible_pages = $vm_new_visible_pages;
+fetch_vacuum_stats();
+
$res = $node->safe_psql($dbname,
- "SELECT vm_new_frozen_pages > $vm_new_frozen_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+ "SELECT vm_new_frozen_pages = $old_vm_new_frozen_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
);
ok($res eq 't', 'vacuum froze some pages after backend activity, as expected') or
fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_frozen_pages', tab_value => $vm_new_frozen_pages,);
$res = $node->safe_psql($dbname,
- "SELECT vm_new_visible_pages > $vm_new_visible_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+ "SELECT vm_new_visible_pages > $old_vm_new_visible_pages FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
);
ok($res eq 't', 'vacuum marked pages all-visible after backend activity, as expected') or
fetch_error_tab_vacuum_statistics(tab_column => 'vm_new_visible_pages', tab_value => $vm_new_visible_pages,);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 7e6029394cb..b627c85e332 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2330,6 +2330,23 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schemaname,
rel.relname,
--
2.39.5 (Apple Git-154)
[text/plain] v26-0004-Vacuum-statistics-have-been-separated-from-regular.patch (71.3K, 6-v26-0004-Vacuum-statistics-have-been-separated-from-regular.patch)
download | inline diff:
From 12f4596c25e15d1f7566b87334615ad112cfb64e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 21 Dec 2025 01:40:06 +0300
Subject: [PATCH 4/5] Vacuum statistics have been separated from regular
relation and database statistics to reduce memory usage. Dedicated
PGSTAT_KIND_VACUUM_RELATION and PGSTAT_KIND_VACUUM_DB entries were added to
the stats collector to efficiently allocate memory for vacuum-specific
metrics, which require significantly more space per relation.
---
src/backend/access/heap/vacuumlazy.c | 124 +++++-----
src/backend/catalog/heap.c | 1 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/system_views.sql | 177 ++++++++-------
src/backend/commands/dbcommands.c | 1 +
src/backend/commands/vacuumparallel.c | 5 +-
src/backend/utils/activity/Makefile | 1 +
src/backend/utils/activity/pgstat.c | 30 ++-
src/backend/utils/activity/pgstat_database.c | 10 +-
src/backend/utils/activity/pgstat_relation.c | 75 +-----
src/backend/utils/activity/pgstat_vacuum.c | 214 ++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 149 ++++++------
src/backend/utils/misc/guc_parameters.dat | 6 +
src/include/commands/vacuum.h | 2 +-
src/include/pgstat.h | 208 +++++++++--------
src/include/utils/pgstat_internal.h | 15 ++
src/include/utils/pgstat_kind.h | 4 +-
.../t/050_vacuum_extending_basic_test.pl | 21 +-
.../t/051_vacuum_extending_freeze_test.pl | 1 +
src/test/regress/expected/rules.out | 146 ++++++------
20 files changed, 735 insertions(+), 456 deletions(-)
create mode 100644 src/backend/utils/activity/pgstat_vacuum.c
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index fcd92a43dda..3f1ed040908 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -413,7 +413,8 @@ typedef struct LVRelState
int32 wraparound_failsafe_count; /* number of emergency vacuums to
* prevent anti-wraparound
* shutdown */
- ExtVacReport extVacReportIdx;
+
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -505,6 +506,9 @@ extvac_stats_start(Relation rel, LVExtStatCounters * counters)
{
TimestampTz starttime;
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
memset(counters, 0, sizeof(LVExtStatCounters));
starttime = GetCurrentTimestamp();
@@ -536,7 +540,7 @@ extvac_stats_start(Relation rel, LVExtStatCounters * counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters * counters,
- ExtVacReport * report)
+ PgStat_CommonCounts * report)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -544,6 +548,11 @@ extvac_stats_end(Relation rel, LVExtStatCounters * counters,
long secs;
int usecs;
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+
/* Calculate diffs of global stat parameters on WAL and buffer usage. */
memset(&walusage, 0, sizeof(WalUsage));
WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
@@ -592,6 +601,9 @@ void
extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx * counters)
{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
/* Set initial values for common heap and index statistics */
extvac_stats_start(rel, &counters->common);
counters->pages_deleted = counters->tuples_removed = 0;
@@ -609,11 +621,15 @@ extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
void
extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx * counters, ExtVacReport * report)
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
{
- memset(report, 0, sizeof(ExtVacReport));
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ extvac_stats_end(rel, &counters->common, &report->common);
- extvac_stats_end(rel, &counters->common, report);
report->type = PGSTAT_EXTVAC_INDEX;
if (stats != NULL)
@@ -624,7 +640,7 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
*/
/* Fill index-specific extended stats fields */
- report->tuples_deleted =
+ report->common.tuples_deleted =
stats->tuples_removed - counters->tuples_removed;
report->index.pages_deleted =
stats->pages_deleted - counters->pages_deleted;
@@ -647,8 +663,11 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
* procudure.
*/
static void
-accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacStats)
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
/* Fill heap-specific extended stats fields */
extVacStats->type = PGSTAT_EXTVAC_TABLE;
extVacStats->table.pages_scanned = vacrel->scanned_pages;
@@ -656,46 +675,45 @@ accumulate_heap_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacStats
extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
- extVacStats->tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
- extVacStats->wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
- extVacStats->blk_read_time -= vacrel->extVacReportIdx.blk_read_time;
- extVacStats->blk_write_time -= vacrel->extVacReportIdx.blk_write_time;
- extVacStats->total_blks_dirtied -= vacrel->extVacReportIdx.total_blks_dirtied;
- extVacStats->total_blks_hit -= vacrel->extVacReportIdx.total_blks_hit;
- extVacStats->total_blks_read -= vacrel->extVacReportIdx.total_blks_read;
- extVacStats->total_blks_written -= vacrel->extVacReportIdx.total_blks_written;
- extVacStats->wal_bytes -= vacrel->extVacReportIdx.wal_bytes;
- extVacStats->wal_fpi -= vacrel->extVacReportIdx.wal_fpi;
- extVacStats->wal_records -= vacrel->extVacReportIdx.wal_records;
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
- extVacStats->total_time -= vacrel->extVacReportIdx.total_time;
- extVacStats->delay_time -= vacrel->extVacReportIdx.delay_time;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
}
static void
-accumulate_idxs_vacuum_statistics(LVRelState *vacrel, ExtVacReport * extVacIdxStats)
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacIdxStats)
{
/* Fill heap-specific extended stats fields */
- vacrel->extVacReportIdx.blk_read_time += extVacIdxStats->blk_read_time;
- vacrel->extVacReportIdx.blk_write_time += extVacIdxStats->blk_write_time;
- vacrel->extVacReportIdx.total_blks_dirtied += extVacIdxStats->total_blks_dirtied;
- vacrel->extVacReportIdx.total_blks_hit += extVacIdxStats->total_blks_hit;
- vacrel->extVacReportIdx.total_blks_read += extVacIdxStats->total_blks_read;
- vacrel->extVacReportIdx.total_blks_written += extVacIdxStats->total_blks_written;
- vacrel->extVacReportIdx.wal_bytes += extVacIdxStats->wal_bytes;
- vacrel->extVacReportIdx.wal_fpi += extVacIdxStats->wal_fpi;
- vacrel->extVacReportIdx.wal_records += extVacIdxStats->wal_records;
- vacrel->extVacReportIdx.delay_time += extVacIdxStats->delay_time;
-
- vacrel->extVacReportIdx.total_time += extVacIdxStats->total_time;
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
}
@@ -856,10 +874,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ErrorContextCallback errcallback;
char **indnames = NULL;
LVExtStatCounters extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Initialize vacuum statistics */
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -915,7 +933,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
- memset(&vacrel->extVacReportIdx, 0, sizeof(ExtVacReport));
+ memset(&vacrel->extVacReportIdx, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
@@ -1173,7 +1192,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
&frozenxid_updated, &minmulti_updated, false);
/* Make generic extended vacuum stats report */
- extvac_stats_end(rel, &extVacCounters, &extVacReport);
+ /* extvac_stats_end(rel, &extVacCounters, &extVacReport.common); */
/*
* Report results to the cumulative stats system, too.
@@ -1190,14 +1209,14 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* Make generic extended vacuum stats report and fill heap-specific
* extended stats fields.
*/
- extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport.common);
accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+ pgstat_report_vacuum_extstats(vacrel->reloid, rel->rd_rel->relisshared, &extVacReport);
pgstat_report_vacuum(rel,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime,
- &extVacReport);
+ starttime);
pgstat_progress_end_command();
if (instrument)
@@ -2902,9 +2921,9 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -2912,7 +2931,7 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
- extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
/*
@@ -3345,9 +3364,9 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
else
{
LVExtStatCounters counters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
- memset(&extVacReport, 0, sizeof(ExtVacReport));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
extvac_stats_start(vacrel->rel, &counters);
@@ -3356,7 +3375,7 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
vacrel->num_index_scans,
estimated_count);
- extvac_stats_end(vacrel->rel, &counters, &extVacReport);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
@@ -3384,7 +3403,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3421,8 +3443,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
if (!ParallelVacuumIsActive(vacrel))
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(indrel,
- 0, 0, 0, &extVacReport);
+ pgstat_report_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -3449,7 +3470,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/* Set initial statistics values to gather vacuum statistics for the index */
extvac_stats_start_idx(indrel, istat, &extVacCounters);
@@ -3485,8 +3506,7 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
if (!ParallelVacuumIsActive(vacrel))
accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
- pgstat_report_vacuum(indrel,
- 0, 0, 0, &extVacReport);
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared, &extVacReport);
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 265cc3e5fbf..680b76b8ef9 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1883,6 +1883,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8dea58ad96b..e906f9e1856 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2327,6 +2327,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index dc86b1ee212..4ec2a7d9f10 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1462,99 +1462,104 @@ GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
--
CREATE VIEW pg_stat_vacuum_tables AS
-SELECT
- ns.nspname AS schemaname,
- rel.relname AS relname,
- stats.relid as relid,
-
- stats.total_blks_read AS total_blks_read,
- stats.total_blks_hit AS total_blks_hit,
- stats.total_blks_dirtied AS total_blks_dirtied,
- stats.total_blks_written AS total_blks_written,
-
- stats.rel_blks_read AS rel_blks_read,
- stats.rel_blks_hit AS rel_blks_hit,
-
- stats.pages_scanned AS pages_scanned,
- stats.pages_removed AS pages_removed,
- stats.vm_new_frozen_pages AS vm_new_frozen_pages,
- stats.vm_new_visible_pages AS vm_new_visible_pages,
- stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
- stats.missed_dead_pages AS missed_dead_pages,
- stats.tuples_deleted AS tuples_deleted,
- stats.tuples_frozen AS tuples_frozen,
- stats.recently_dead_tuples AS recently_dead_tuples,
- stats.missed_dead_tuples AS missed_dead_tuples,
-
- stats.wraparound_failsafe AS wraparound_failsafe,
- stats.index_vacuum_count AS index_vacuum_count,
- stats.wal_records AS wal_records,
- stats.wal_fpi AS wal_fpi,
- stats.wal_bytes AS wal_bytes,
-
- stats.blk_read_time AS blk_read_time,
- stats.blk_write_time AS blk_write_time,
-
- stats.delay_time AS delay_time,
- stats.total_time AS total_time
-
-FROM pg_class rel
- JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
- LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
-WHERE rel.relkind = 'r';
+ SELECT
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ S.relid as relid,
+
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
+
+ S.pages_scanned AS pages_scanned,
+ S.pages_removed AS pages_removed,
+ S.vm_new_frozen_pages AS vm_new_frozen_pages,
+ S.vm_new_visible_pages AS vm_new_visible_pages,
+ S.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ S.missed_dead_pages AS missed_dead_pages,
+ S.tuples_deleted AS tuples_deleted,
+ S.tuples_frozen AS tuples_frozen,
+ S.recently_dead_tuples AS recently_dead_tuples,
+ S.missed_dead_tuples AS missed_dead_tuples,
+
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.index_vacuum_count AS index_vacuum_count,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+
+ FROM pg_class C JOIN
+ pg_namespace N ON N.oid = C.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(C.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
CREATE VIEW pg_stat_vacuum_indexes AS
-SELECT
- rel.oid as relid,
- ns.nspname AS schemaname,
- rel.relname AS relname,
+ SELECT
+ C.oid AS relid,
+ I.oid AS indexrelid,
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ I.relname AS indexrelname,
- total_blks_read AS total_blks_read,
- total_blks_hit AS total_blks_hit,
- total_blks_dirtied AS total_blks_dirtied,
- total_blks_written AS total_blks_written,
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
- rel_blks_read AS rel_blks_read,
- rel_blks_hit AS rel_blks_hit,
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
- pages_deleted AS pages_deleted,
- tuples_deleted AS tuples_deleted,
+ S.pages_deleted AS pages_deleted,
+ S.tuples_deleted AS tuples_deleted,
- wal_records AS wal_records,
- wal_fpi AS wal_fpi,
- wal_bytes AS wal_bytes,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
- blk_read_time AS blk_read_time,
- blk_write_time AS blk_write_time,
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
- delay_time AS delay_time,
- total_time AS total_time
-FROM
- pg_class rel
- JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
- LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+ FROM
+ pg_class C JOIN
+ pg_index X ON C.oid = X.indrelid JOIN
+ pg_class I ON I.oid = X.indexrelid
+ LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace),
+ LATERAL pg_stat_get_vacuum_indexes(I.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
CREATE VIEW pg_stat_vacuum_database AS
-SELECT
- db.oid as dboid,
- db.datname AS dbname,
-
- stats.db_blks_read AS db_blks_read,
- stats.db_blks_hit AS db_blks_hit,
- stats.total_blks_dirtied AS total_blks_dirtied,
- stats.total_blks_written AS total_blks_written,
-
- stats.wal_records AS wal_records,
- stats.wal_fpi AS wal_fpi,
- stats.wal_bytes AS wal_bytes,
-
- stats.blk_read_time AS blk_read_time,
- stats.blk_write_time AS blk_write_time,
-
- stats.delay_time AS delay_time,
- stats.total_time AS total_time,
- stats.wraparound_failsafe AS wraparound_failsafe
-FROM
- pg_database db,
- LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
+ SELECT
+ D.oid as dboid,
+ D.datname AS dbname,
+
+ S.db_blks_read AS db_blks_read,
+ S.db_blks_hit AS db_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time,
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.errors AS errors
+ FROM
+ pg_database D,
+ LATERAL pg_stat_get_vacuum_database(D.oid) S;
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index d1f3be89b35..bf3cd3b1cc9 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1815,6 +1815,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 43450685b09..c7dd2bb52f6 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,7 +869,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
LVExtStatCountersIdx extVacCounters;
- ExtVacReport extVacReport;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -911,8 +911,7 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
/* Make extended vacuum stats report for index */
extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
- pgstat_report_vacuum(indrel,
- 0, 0, 0, &extVacReport);
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared, &extVacReport);
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 9c2443e1ecd..183f7514d2d 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index f317c6e8e90..cdc9cab01cf 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = false;
/* ----------
* state shared with pgstat_*.c
@@ -482,6 +482,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 65207d30378..80e6c7c229a 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
@@ -485,7 +494,6 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
- memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 2675c541369..e8665d23099 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
- bool accumulate_reltype_specific_info);
/*
@@ -210,12 +208,11 @@ pgstat_drop_relation(Relation rel)
*/
void
pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
- PgStat_Counter deadtuples, TimestampTz starttime, ExtVacReport * params)
+ PgStat_Counter deadtuples, TimestampTz starttime)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
- PgStatShared_Database *dbentry;
Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
PgStat_Counter elapsedtime;
@@ -237,8 +234,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
- pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
-
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -274,16 +269,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
*/
pgstat_flush_io(false);
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
-
- if (dboid != InvalidOid)
- {
- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
- dboid, InvalidOid, false);
- dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
-
- pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
- pgstat_unlock_entry(entry_ref);
- }
}
/*
@@ -918,6 +903,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
@@ -1027,55 +1018,3 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
-
-static void
-pgstat_accumulate_extvac_stats(ExtVacReport * dst, ExtVacReport * src,
- bool accumulate_reltype_specific_info)
-{
- dst->total_blks_read += src->total_blks_read;
- dst->total_blks_hit += src->total_blks_hit;
- dst->total_blks_dirtied += src->total_blks_dirtied;
- dst->total_blks_written += src->total_blks_written;
- dst->wal_bytes += src->wal_bytes;
- dst->wal_fpi += src->wal_fpi;
- dst->wal_records += src->wal_records;
- dst->blk_read_time += src->blk_read_time;
- dst->blk_write_time += src->blk_write_time;
- dst->delay_time += src->delay_time;
- dst->total_time += src->total_time;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
-
- if (!accumulate_reltype_specific_info)
- return;
-
- if (dst->type == PGSTAT_EXTVAC_INVALID)
- dst->type = src->type;
-
- Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
-
- if (dst->type == src->type)
- {
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- if (dst->type == PGSTAT_EXTVAC_TABLE)
- {
- dst->table.pages_scanned += src->table.pages_scanned;
- dst->table.pages_removed += src->table.pages_removed;
- dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
- dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
- dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->table.tuples_frozen += src->table.tuples_frozen;
- dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
- dst->table.index_vacuum_count += src->table.index_vacuum_count;
- dst->table.missed_dead_pages += src->table.missed_dead_pages;
- dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
- }
- else if (dst->type == PGSTAT_EXTVAC_INDEX)
- {
- dst->index.pages_deleted += src->index.pages_deleted;
- dst->tuples_deleted += src->tuples_deleted;
- }
- }
-}
diff --git a/src/backend/utils/activity/pgstat_vacuum.c b/src/backend/utils/activity/pgstat_vacuum.c
new file mode 100644
index 00000000000..340ee24f26a
--- /dev/null
+++ b/src/backend/utils/activity/pgstat_vacuum.c
@@ -0,0 +1,214 @@
+#include "postgres.h"
+
+#include "pgstat.h"
+#include "utils/pgstat_internal.h"
+#include "utils/memutils.h"
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+bool pgstat_track_vacuum_statistics_for_relations = false;
+
+#define ACCUMULATE_FIELD(field) dst->field += src->field;
+
+#define ACCUMULATE_SUBFIELD(substruct, field) \
+ (dst->substruct.field += src->substruct.field)
+
+static void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUMULATE_FIELD(total_blks_read);
+ ACCUMULATE_FIELD(total_blks_hit);
+ ACCUMULATE_FIELD(total_blks_dirtied);
+ ACCUMULATE_FIELD(total_blks_written);
+
+ ACCUMULATE_FIELD(blks_fetched);
+ ACCUMULATE_FIELD(blks_hit);
+
+ ACCUMULATE_FIELD(wal_records);
+ ACCUMULATE_FIELD(wal_fpi);
+ ACCUMULATE_FIELD(wal_bytes);
+
+ ACCUMULATE_FIELD(blk_read_time);
+ ACCUMULATE_FIELD(blk_write_time);
+ ACCUMULATE_FIELD(delay_time);
+ ACCUMULATE_FIELD(total_time);
+
+ ACCUMULATE_FIELD(tuples_deleted);
+ ACCUMULATE_FIELD(wraparound_failsafe_count);
+}
+
+static void
+pgstat_accumulate_extvac_stats_relations(PgStat_VacuumRelationCounts * dst, PgStat_VacuumRelationCounts * src)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB && src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ ACCUMULATE_SUBFIELD(common, blks_fetched);
+ ACCUMULATE_SUBFIELD(common, blks_hit);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(table, pages_scanned);
+ ACCUMULATE_SUBFIELD(table, pages_removed);
+ ACCUMULATE_SUBFIELD(table, vm_new_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, tuples_frozen);
+ ACCUMULATE_SUBFIELD(table, recently_dead_tuples);
+ ACCUMULATE_SUBFIELD(table, index_vacuum_count);
+ ACCUMULATE_SUBFIELD(table, missed_dead_pages);
+ ACCUMULATE_SUBFIELD(table, missed_dead_tuples);
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(index, pages_deleted);
+ }
+}
+
+static void
+pgstat_accumulate_extvac_stats_db(PgStat_VacuumDBCounts * dst, PgStat_VacuumDBCounts * src)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+}
+
+/*
+ * Report that the table was just vacuumed and flush statistics.
+ */
+void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_VacuumRelation *shtabentry;
+ PgStatShared_VacuumDB *shdbentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_RELATION,
+ dboid, tableoid, false);
+ shtabentry = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+ pgstat_accumulate_extvac_stats_relations(&shtabentry->stats, params);
+
+ pgstat_unlock_entry(entry_ref);
+
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_DB,
+ dboid, InvalidOid, false);
+
+ shdbentry = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ pgstat_accumulate_common(&shdbentry->stats.common, ¶ms->common);
+
+ pgstat_unlock_entry(entry_ref);
+}
+
+/*
+ * Flush out pending stats for the entry
+ *
+ * If nowait is true, this function returns false if lock could not
+ * immediately acquired, otherwise true is returned.
+ */
+bool
+pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumRelation *shtabstats;
+ PgStat_RelationVacuumPending *pendingent; /* table entry of shared stats */
+
+ pendingent = (PgStat_RelationVacuumPending *) entry_ref->pending;
+ shtabstats = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts.
+ */
+ if (pg_memory_is_all_zeros(&pendingent,
+ sizeof(struct PgStat_RelationVacuumPending)))
+ return true;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ {
+ return false;
+ }
+
+ pgstat_accumulate_extvac_stats_relations(&(shtabstats->stats), &(pendingent->counts));
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the vacuum collected statistics for one relation or NULL.
+ */
+PgStat_VacuumRelationCounts *
+pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid)
+{
+ return (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_RELATION, dbid, relid);
+}
+
+PgStat_VacuumDBCounts *
+pgstat_fetch_stat_vacuum_dbentry(Oid dbid)
+{
+ return (PgStat_VacuumDBCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_DB, dbid, InvalidOid);
+}
+
+bool
+pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumDB *sharedent;
+ PgStat_VacuumDBCounts *pendingent;
+
+ pendingent = (PgStat_VacuumDBCounts *) entry_ref->pending;
+ sharedent = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ /* The entry was successfully flushed, add the same to database stats */
+ pgstat_accumulate_extvac_stats_db(&(sharedent->stats), pendingent);
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Find or create a local PgStat_VacuumDBCounts entry for dboid.
+ */
+PgStat_VacuumDBCounts *
+pgstat_prep_vacuum_database_pending(Oid dboid)
+{
+ PgStat_EntryRef *entry_ref;
+
+ /*
+ * This should not report stats on database objects before having
+ * connected to a database.
+ */
+ Assert(!OidIsValid(dboid) || OidIsValid(MyDatabaseId));
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_VACUUM_DB, dboid, InvalidOid,
+ NULL);
+
+ if (entry_ref == NULL)
+ return NULL;
+
+ return entry_ref->pending;
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 4e2714f2e6a..0a64f034a3f 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2314,7 +2314,6 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
-
/*
* Get the vacuum statistics for the heap tables.
*/
@@ -2324,41 +2323,45 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
char buf[256];
int i = 0;
+ /* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
- tabentry = pgstat_fetch_stat_tabentry(relid);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- if (!tabentry)
- {
- InitMaterializedSRF(fcinfo, 0);
- PG_RETURN_VOID();
- }
- else
+ if (!pending)
{
- extvacuum = &(tabentry->vacuum_ext);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, 0);
+
+ if (!pending)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
}
+ extvacuum = pending;
+
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
@@ -2366,28 +2369,28 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
@@ -2404,8 +2407,8 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
Oid relid = PG_GETARG_OID(0);
- PgStat_StatTabEntry *tabentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
@@ -2415,48 +2418,51 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
- tabentry = pgstat_fetch_stat_tabentry(relid);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
- if (tabentry == NULL)
- {
- InitMaterializedSRF(fcinfo, 0);
- PG_RETURN_VOID();
- }
- else
+ if (!pending)
{
- extvacuum = &(tabentry->vacuum_ext);
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, 0);
+
+ if (!pending)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
}
+ extvacuum = pending;
+
i = 0;
values[i++] = ObjectIdGetDatum(relid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->blks_fetched -
- extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
@@ -2470,8 +2476,8 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
Oid dbid = PG_GETARG_OID(0);
- PgStat_StatDBEntry *dbentry;
- ExtVacReport *extvacuum;
+ PgStat_VacuumDBCounts *extvacuum;
+ PgStat_VacuumDBCounts *pending;
TupleDesc tupdesc;
Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
@@ -2481,42 +2487,41 @@ pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
- dbentry = pgstat_fetch_stat_dbentry(dbid);
+ pending = pgstat_fetch_stat_vacuum_dbentry(dbid);
- if (dbentry == NULL)
+ if (!pending)
{
InitMaterializedSRF(fcinfo, 0);
PG_RETURN_VOID();
}
- else
- {
- extvacuum = &(dbentry->vacuum_ext);
- }
+
+ extvacuum = pending;
i = 0;
values[i++] = ObjectIdGetDatum(dbid);
- values[i++] = Int64GetDatum(extvacuum->total_blks_read);
- values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
- values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
- values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
- values[i++] = Int64GetDatum(extvacuum->wal_records);
- values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
/* Convert to numeric, like pg_stat_statements */
- snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
values[i++] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
Int32GetDatum(-1));
- values[i++] = Float8GetDatum(extvacuum->blk_read_time);
- values[i++] = Float8GetDatum(extvacuum->blk_write_time);
- values[i++] = Float8GetDatum(extvacuum->delay_time);
- values[i++] = Float8GetDatum(extvacuum->total_time);
- values[i++] = Int32GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->errors);
Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 3b9d8349078..631df3a57c3 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -3084,6 +3084,12 @@
boot_val => 'false',
},
+{ name => 'track_vacuum_statistics', type => 'bool', context => 'PGC_SUSET', group => 'STATS_CUMULATIVE',
+ short_desc => 'Collects vacuum statistics for vacuum activity.',
+ variable => 'pgstat_track_vacuum_statistics',
+ boot_val => 'false',
+},
+
{ name => 'track_wal_io_timing', type => 'bool', context => 'PGC_SUSET', group => 'STATS_CUMULATIVE',
short_desc => 'Collects timing statistics for WAL I/O activity.',
variable => 'track_wal_io_timing',
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index b48ace6084b..6e85b08aa89 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -437,5 +437,5 @@ extern double anl_get_next_S(double t, int n, double *stateptr);
extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
LVExtStatCountersIdx * counters);
extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
- LVExtStatCountersIdx * counters, ExtVacReport * report);
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f3bdc1c38df..61d488f1bf8 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -119,54 +119,100 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_TABLE = 1,
- PGSTAT_EXTVAC_INDEX = 2
-} ExtVacReportType;
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
/* ----------
+ * PgStat_TableCounts The actual per-table counts kept by a backend
*
- * ExtVacReport
+ * This struct should contain only actual event counters, because we make use
+ * of pg_memory_is_all_zeros() to detect whether there are any stats updates
+ * to apply.
*
- * Additional statistics of vacuum processing over a relation.
- * pages_removed is the amount by which the physically shrank,
- * if any (ie the change in its total size on disk)
- * pages_deleted refer to free space within the index file
+ * It is a component of PgStat_TableStatus (within-backend state).
+ *
+ * Note: for a table, tuples_returned is the number of tuples successfully
+ * fetched by heap_getnext, while tuples_fetched is the number of tuples
+ * successfully fetched by heap_fetch under the control of bitmap indexscans.
+ * For an index, tuples_returned is the number of index entries returned by
+ * the index AM, while tuples_fetched is the number of tuples successfully
+ * fetched by heap_fetch under the control of simple indexscans for this index.
+ *
+ * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
+ * actions, regardless of whether the transaction committed. delta_live_tuples,
+ * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
+ * Note that delta_live_tuples and delta_dead_tuples can be negative!
* ----------
*/
-typedef struct ExtVacReport
+typedef struct PgStat_TableCounts
{
- /*
- * number of blocks missed, hit, dirtied and written during a vacuum of
- * specific relation
- */
+ PgStat_Counter numscans;
+
+ PgStat_Counter tuples_returned;
+ PgStat_Counter tuples_fetched;
+
+ PgStat_Counter tuples_inserted;
+ PgStat_Counter tuples_updated;
+ PgStat_Counter tuples_deleted;
+ PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_newpage_updated;
+ bool truncdropped;
+
+ PgStat_Counter delta_live_tuples;
+ PgStat_Counter delta_dead_tuples;
+ PgStat_Counter changed_tuples;
+
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+} PgStat_TableCounts;
+
+typedef struct PgStat_CommonCounts
+{
+ /* blocks */
int64 total_blks_read;
int64 total_blks_hit;
int64 total_blks_dirtied;
int64 total_blks_written;
- /*
- * blocks missed and hit for just the heap during a vacuum of specific
- * relation
- */
+ /* heap blocks */
int64 blks_fetched;
int64 blks_hit;
- /* Vacuum WAL usage stats */
- int64 wal_records; /* wal usage: number of WAL records */
- int64 wal_fpi; /* wal usage: number of WAL full page images
- * produced */
- uint64 wal_bytes; /* wal usage: size of WAL records produced */
+ /* WAL */
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
- /* Time stats. */
- double blk_read_time; /* time spent reading pages, in msec */
- double blk_write_time; /* time spent writing pages, in msec */
- double delay_time; /* how long vacuum slept in vacuum delay
- * point, in msec */
- double total_time; /* total time of a vacuum operation, in msec */
+ /* Time */
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
- int64 tuples_deleted; /* tuples deleted by vacuum */
+ /* tuples */
+ int64 tuples_deleted;
- int32 wraparound_failsafe_count; /* the number of times to prevent
- * wraparound problem */
+ /* failsafe */
+ int32 wraparound_failsafe_count;
+} PgStat_CommonCounts;
+
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
ExtVacReportType type; /* heap, index, etc. */
@@ -185,6 +231,13 @@ typedef struct ExtVacReport
{
struct
{
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are
+ * still visible to some
+ * transaction */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due
+ * to failure to get a cleanup
+ * lock */
int64 pages_scanned; /* heap pages examined (not skipped by
* VM) */
int64 pages_removed; /* heap pages removed by vacuum
@@ -192,10 +245,6 @@ typedef struct ExtVacReport
int64 pages_frozen; /* pages marked in VM as frozen */
int64 pages_all_visible; /* pages marked in VM as
* all-visible */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are
- * still visible to some
- * transaction */
int64 vm_new_frozen_pages; /* pages marked in VM as
* frozen */
int64 vm_new_visible_pages; /* pages marked in VM as
@@ -203,9 +252,6 @@ typedef struct ExtVacReport
int64 vm_new_visible_frozen_pages; /* pages marked in VM as
* all-visible and
* frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due
- * to failure to get a cleanup
- * lock */
int64 missed_dead_pages; /* pages with missed dead tuples */
int64 index_vacuum_count; /* number of index vacuumings */
} table;
@@ -214,60 +260,21 @@ typedef struct ExtVacReport
int64 pages_deleted; /* number of pages deleted by vacuum */
} index;
} /* per_type_stats */ ;
-} ExtVacReport;
+} PgStat_VacuumRelationCounts;
-/* ----------
- * PgStat_TableCounts The actual per-table counts kept by a backend
- *
- * This struct should contain only actual event counters, because we make use
- * of pg_memory_is_all_zeros() to detect whether there are any stats updates
- * to apply.
- *
- * It is a component of PgStat_TableStatus (within-backend state).
- *
- * Note: for a table, tuples_returned is the number of tuples successfully
- * fetched by heap_getnext, while tuples_fetched is the number of tuples
- * successfully fetched by heap_fetch under the control of bitmap indexscans.
- * For an index, tuples_returned is the number of index entries returned by
- * the index AM, while tuples_fetched is the number of tuples successfully
- * fetched by heap_fetch under the control of simple indexscans for this index.
- *
- * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted
- * actions, regardless of whether the transaction committed. delta_live_tuples,
- * delta_dead_tuples, and changed_tuples are set depending on commit or abort.
- * Note that delta_live_tuples and delta_dead_tuples can be negative!
- * ----------
- */
-typedef struct PgStat_TableCounts
+typedef struct PgStat_VacuumRelationStatus
{
- PgStat_Counter numscans;
-
- PgStat_Counter tuples_returned;
- PgStat_Counter tuples_fetched;
-
- PgStat_Counter tuples_inserted;
- PgStat_Counter tuples_updated;
- PgStat_Counter tuples_deleted;
- PgStat_Counter tuples_hot_updated;
- PgStat_Counter tuples_newpage_updated;
- bool truncdropped;
-
- PgStat_Counter delta_live_tuples;
- PgStat_Counter delta_dead_tuples;
- PgStat_Counter changed_tuples;
-
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-
- PgStat_Counter rev_all_visible_pages;
- PgStat_Counter rev_all_frozen_pages;
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
- /*
- * Additional cumulative stat on vacuum operations. Use an expensive
- * structure as an abstraction for different types of relations.
- */
- ExtVacReport vacuum_ext;
-} PgStat_TableCounts;
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ PgStat_CommonCounts common;
+ int32 errors;
+} PgStat_VacuumDBCounts;
/* ----------
* PgStat_TableStatus Per-table status within a backend
@@ -293,6 +300,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -489,8 +502,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
-
- ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -578,8 +589,6 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter rev_all_visible_pages;
PgStat_Counter rev_all_frozen_pages;
-
- ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
/* ------
@@ -788,7 +797,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
- TimestampTz starttime, ExtVacReport * params);
+ TimestampTz starttime);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
@@ -924,6 +933,16 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+ pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts * pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts * pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid);
+PgStat_VacuumDBCounts *pgstat_fetch_stat_vacuum_dbentry(Oid dbid);
+
/*
* Functions in pgstat_wal.c
*/
@@ -940,7 +959,8 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
-
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics_for_relations;
/*
* Variables in pgstat_bgwriter.c
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 7dffab8dbdd..4abe70cb54e 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -500,6 +500,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -678,6 +690,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index eb5f0b3ae6d..52e884fbf8b 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
diff --git a/src/test/recovery/t/050_vacuum_extending_basic_test.pl b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
index bd3cb544e30..e2fd541fd89 100644
--- a/src/test/recovery/t/050_vacuum_extending_basic_test.pl
+++ b/src/test/recovery/t/050_vacuum_extending_basic_test.pl
@@ -28,6 +28,7 @@ $node->init;
# Configure the server logging level for the test
$node->append_conf('postgresql.conf', q{
log_min_messages = notice
+ track_vacuum_statistics = on
});
my $stderr;
@@ -64,8 +65,9 @@ $node->safe_psql($dbname, q{
$node->safe_psql(
$dbname,
- "CREATE TABLE vestat (x int PRIMARY KEY)
+ "CREATE TABLE vestat (x int)
WITH (autovacuum_enabled = off, fillfactor = 10);
+ create index vestat_pkey on vestat (x);
INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
ANALYZE vestat;"
);
@@ -115,7 +117,7 @@ sub wait_for_vacuum_stats {
AND
(SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
FROM pg_stat_vacuum_indexes
- WHERE relname = 'vestat_pkey');"
+ WHERE indexrelname = 'vestat_pkey');"
);
return 1 if ($result_query eq 't');
@@ -183,7 +185,7 @@ sub fetch_vacuum_stats {
$dbname,
"SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
FROM pg_stat_vacuum_indexes
- WHERE relname = 'vestat_pkey';"
+ WHERE indexrelname = 'vestat_pkey';"
);
$index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
@@ -321,7 +323,7 @@ sub fetch_error_base_idx_vacuum_statistics {
$dbname,
"SELECT tuples_deleted, pages_deleted
FROM pg_stat_vacuum_indexes
- WHERE relname = 'vestat_pkey';"
+ WHERE indexrelname = 'vestat_pkey';"
);
$base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
my ($cur_tuples_deleted, $cur_pages_deleted) = split /\s+/, $base_statistics;
@@ -343,7 +345,7 @@ sub fetch_error_wal_idx_vacuum_statistics {
$dbname,
"SELECT wal_records, wal_bytes, wal_fpi
FROM pg_stat_vacuum_indexes
- WHERE relname = 'vestat_pkey';"
+ WHERE indexrelname = 'vestat_pkey';"
);
$wal_raw =~ s/\s*\|\s*/ /g; # transform " | " in space
@@ -707,7 +709,7 @@ $base_stats = $node->safe_psql(
'postgres',
"SELECT count(*) = 0
FROM pg_stat_vacuum_indexes
- WHERE relname = 'vestat_pkey';"
+ WHERE indexrelname = 'vestat_pkey';"
);
ok($base_stats eq 't', 'check the printing index vacuum extended statistics from another database are not available');
@@ -742,6 +744,9 @@ $reloid = $node->safe_psql(
}
);
+# Run VACUUM on shared table to ensure stats entry is created
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
# Check if we can get vacuum statistics for cluster relations (dbid = 0)
$base_stats = $node->safe_psql(
$dbname,
@@ -760,6 +765,10 @@ my $indoid = $node->safe_psql(
}
);
+# Run VACUUM on shared index to ensure stats entry is created
+# Note: VACUUM on the table will also vacuum its indexes
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
$base_stats = $node->safe_psql(
$dbname,
qq{
diff --git a/src/test/recovery/t/051_vacuum_extending_freeze_test.pl b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
index 7528f20098b..2a1c506a22f 100644
--- a/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
+++ b/src/test/recovery/t/051_vacuum_extending_freeze_test.pl
@@ -37,6 +37,7 @@ $node->append_conf('postgresql.conf', q{
vacuum_max_eager_freeze_failure_rate = 1.0
vacuum_failsafe_age = 0
vacuum_multixact_failsafe_age = 0
+ track_vacuum_statistics = on
});
$node->start();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index b627c85e332..4e8b8b8a2b1 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2330,77 +2330,81 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
-pg_stat_vacuum_database| SELECT db.oid AS dboid,
- db.datname AS dbname,
- stats.db_blks_read,
- stats.db_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time,
- stats.wraparound_failsafe,
- stats.errors
- FROM pg_database db,
- LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
-pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
- ns.nspname AS schemaname,
- rel.relname,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_deleted,
- stats.tuples_deleted,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time
- FROM (pg_class rel
- JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
- LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
- WHERE (rel.relkind = 'i'::"char");
-pg_stat_vacuum_tables| SELECT ns.nspname AS schemaname,
- rel.relname,
- stats.relid,
- stats.total_blks_read,
- stats.total_blks_hit,
- stats.total_blks_dirtied,
- stats.total_blks_written,
- stats.rel_blks_read,
- stats.rel_blks_hit,
- stats.pages_scanned,
- stats.pages_removed,
- stats.vm_new_frozen_pages,
- stats.vm_new_visible_pages,
- stats.vm_new_visible_frozen_pages,
- stats.missed_dead_pages,
- stats.tuples_deleted,
- stats.tuples_frozen,
- stats.recently_dead_tuples,
- stats.missed_dead_tuples,
- stats.wraparound_failsafe,
- stats.index_vacuum_count,
- stats.wal_records,
- stats.wal_fpi,
- stats.wal_bytes,
- stats.blk_read_time,
- stats.blk_write_time,
- stats.delay_time,
- stats.total_time
- FROM (pg_class rel
- JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
- LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
- WHERE (rel.relkind = 'r'::"char");
+pg_stat_vacuum_database| SELECT d.oid AS dboid,
+ d.datname AS dbname,
+ s.db_blks_read,
+ s.db_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time,
+ s.wraparound_failsafe,
+ s.errors
+ FROM pg_database d,
+ LATERAL pg_stat_get_vacuum_database(d.oid) s(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
+pg_stat_vacuum_indexes| SELECT c.oid AS relid,
+ i.oid AS indexrelid,
+ n.nspname AS schemaname,
+ c.relname,
+ i.relname AS indexrelname,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_deleted,
+ s.tuples_deleted,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (((pg_class c
+ JOIN pg_index x ON ((c.oid = x.indrelid)))
+ JOIN pg_class i ON ((i.oid = x.indexrelid)))
+ LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(i.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
+pg_stat_vacuum_tables| SELECT n.nspname AS schemaname,
+ c.relname,
+ s.relid,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_scanned,
+ s.pages_removed,
+ s.vm_new_frozen_pages,
+ s.vm_new_visible_pages,
+ s.vm_new_visible_frozen_pages,
+ s.missed_dead_pages,
+ s.tuples_deleted,
+ s.tuples_frozen,
+ s.recently_dead_tuples,
+ s.missed_dead_tuples,
+ s.wraparound_failsafe,
+ s.index_vacuum_count,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (pg_class c
+ JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(c.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
--
2.39.5 (Apple Git-154)
[text/plain] v26-0005-Add-documentation-about-the-system-views-that-are-us.patch (24.5K, 7-v26-0005-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From cfbf50c85ae9bfdfb9167dab446bf3256e33ddb1 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 5/5] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 755 +++++++++++++++++++++++++++++++++
1 file changed, 755 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 162c76b729a..50578cae90d 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5654,4 +5654,759 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-09 15:46 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2026-03-09 15:46 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
I discovered that my last patches were incorrectly formed. I updated the
correct version.
On 09.03.2026 18:25, Alena Rybakina wrote:
> I developed a patch with a different approach - using custom
> statistics. As in the previous approach, I store statistics separately
> in slots for relations and for databases. It was also necessary to
> introduce a hook, and all control is handled through an extension.
> Statistics collection in the core occurs only if the hook is defined
> (i.e., the extension is added to shared_preload_libraries).
> The extension is also controlled by additional gucs that allow
> disabling vacuum statistics collection, or collecting statistics only
> for system relations, only for user relations, or only at the database
> or relation level.
>
> For now, I have divided them into several categories: general
> statistics (including the number of removed tables and tuples, and how
> many times wraparound prevention occurred), cost-based statistics,
> buffer statistics, and timing statistics. Memory is dynamically freed
> or allocated when the corresponding guc configuration changes. This
> approach is still a work in progress.
>
> In the README and documentation in the extension, I also added
> information about how much memory will be used to store the objects
> (approximately 300 KB) and added the function to measure memory
> consumption.
>
> Currently there are three patches:
> The first patch still collects statistics about frozen pages and pages
> where the visibility flag is cleared.
> The second patch implements statistics collection in the core, but
> without storing them.
>
> The third patch is the extension itself, where the statistics are
> stored and displayed, with the control mechanisms described above.
>
>
> -----------
> Best regards,
> Alena Rybakina
From 7811d1b76b8e65c0eb364c8d113df7a304422a8a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 18:30:12 +0300
Subject: [PATCH 1/3] Introduce new statistics tracking the number of times the
all-visible and all-frozen bits are cleared in the visibility map
(rev_all_visible_pages and rev_all_frozen_pages). These counters, together
with the existing per-vacuum frozen page statistics (vm_new_frozen_pages,
vm_new_visible_pages), help assess how aggressively vacuum is configured and
how frequently the backend has to revoke all-frozen/all-visible bits due to
concurrent modifications.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/visibilitymap.c | 10 ++++++++++
src/backend/catalog/system_views.sql | 4 +++-
src/backend/utils/activity/pgstat_relation.c | 2 ++
src/backend/utils/adt/pgstatfuncs.c | 6 ++++++
src/include/catalog/pg_proc.dat | 12 +++++++++++-
src/include/pgstat.h | 18 +++++++++++++++++-
src/test/regress/expected/rules.out | 12 +++++++++---
7 files changed, 58 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..9ea7a068ef0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ecb7c996e86..1242eca7304 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -741,7 +741,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..885d590d2b2 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 50ea9e8fb83..83ff1fff87d 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..252eab079d6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12831,6 +12831,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..02fbb8480dd 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,9 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -466,6 +469,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autoanalyze_time;
TimestampTz stat_reset_time;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_StatTabEntry;
/* ------
@@ -725,6 +730,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index deb6e2ad6a9..e392428377d 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2279,7 +2281,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2334,7 +2338,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
From f78253895ef7e489e59389d60bc596b7cf42a19e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
Vacuum statistics are stored separately from regular relation and
database statistics. Dedicated PGSTAT_KIND_VACUUM_RELATION and
PGSTAT_KIND_VACUUM_DB entries in the cumulative statistics system
allocate memory for vacuum-specific metrics. Statistics are gathered
separately for tables and indexes according to vacuum phases. The
ExtVacReport union and type field distinguish PGSTAT_EXTVAC_TABLE vs
PGSTAT_EXTVAC_INDEX. Heap vacuum stats are sent to the cumulative
statistics system after vacuum has processed the indexes. Database
vacuum statistics aggregate per-table and per-index statistics within
the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 140 ++++++++++++++++++++++++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 8 ++
src/include/commands/vacuum.h | 28 ++++++
src/include/pgstat.h | 58 +++++++++++
5 files changed, 238 insertions(+)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 82c5b28e0ad..04b087e2a5c 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -282,6 +282,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -402,6 +404,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -488,6 +499,107 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -646,7 +758,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params.log_vacuum_min_duration >= 0));
@@ -663,6 +778,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -690,7 +807,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -801,6 +920,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -977,6 +1099,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
starttime);
+
pgstat_progress_end_command();
if (instrument)
@@ -3018,6 +3141,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3129,6 +3253,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3147,6 +3275,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3155,6 +3284,9 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3179,6 +3311,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3198,12 +3334,16 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 62c1ebdfd9b..faeab06d2bc 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2541,6 +2544,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 279108ca89f..7a85c644749 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,6 +869,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -877,6 +879,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -905,6 +909,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1055,6 +1062,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index e885a4b9c77..c50ce51e9da 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -333,6 +334,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 02fbb8480dd..7fe8e5468b8 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -92,6 +92,63 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -680,6 +737,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
From 7c42b68e7ebeeceb1475502d2ab5e2f9bd543670 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 3 Mar 2026 00:17:13 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics extension for extended vacuum
statistics.
This commit introduces the ext_vacuum_statistics extension, which provides
extended vacuum statistics through a dedicated schema and views. Statistics
are stored via the pgstat custom statistics infrastructure. The extension
registers set_report_vacuum_hook to receive vacuum metrics and persists
them into custom stats; when the hook is not set, no additional overhead
is incurred.
Views pg_stats_vacuum_tables, pg_stats_vacuum_indexes and
pg_stats_vacuum_database expose per-table, per-index and aggregated
per-database vacuum statistics respectively.
GUCs control which objects are tracked and how. vacuum_statistics.enabled
(default on) turns collection on or off. vacuum_statistics.object_types
(default all) restricts tracking to databases only, relations only, or both.
When tracking relations, vacuum_statistics.track_relations (default all)
filters by system or user tables. vacuum_statistics.track_databases_from_list
and vacuum_statistics.track_relations_from_list (both default off) restrict
tracking to databases and relations explicitly added via add_track_database
and add_track_relation; when off, all objects of the chosen types are tracked.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 +++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 260 +++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 +++++
.../t/054_vacuum_extending_gucs_test.pl | 203 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1000 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 +++++++++
doc/src/sgml/filelist.sgml | 1 +
src/backend/access/heap/vacuumlazy.c | 112 +-
src/backend/commands/vacuumparallel.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 24 +
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 11 +
23 files changed, 3576 insertions(+), 18 deletions(-)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 2f0a88d3f77..6e064c566aa 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..4f0b1877f90
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,260 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT VOLATILE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT VOLATILE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT VOLATILE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..0ba52f7988f
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_rev_all_frozen_pages(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_rev_all_visible_pages(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..b8d5bf30ecf
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,203 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra));
+ $node->safe_psql($db, $sql);
+ sleep(0.1);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no stats when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..1f6f3e90614
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1000 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+
+/* Hash tables for track_databases and track_relations_list */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+static void evs_track_load_file(void);
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+#define ACCUM_IF(dst, src, field) \
+ do { (dst)->field += (src)->field; } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read);
+ ACCUM_IF(dst, src, total_blks_hit);
+ ACCUM_IF(dst, src, total_blks_dirtied);
+ ACCUM_IF(dst, src, total_blks_written);
+ ACCUM_IF(dst, src, blks_fetched);
+ ACCUM_IF(dst, src, blks_hit);
+ ACCUM_IF(dst, src, blk_read_time);
+ ACCUM_IF(dst, src, blk_write_time);
+ ACCUM_IF(dst, src, delay_time);
+ ACCUM_IF(dst, src, total_time);
+ ACCUM_IF(dst, src, wal_records);
+ ACCUM_IF(dst, src, wal_fpi);
+ ACCUM_IF(dst, src, wal_bytes);
+ ACCUM_IF(dst, src, wraparound_failsafe_count);
+ ACCUM_IF(dst, src, interrupts_count);
+ ACCUM_IF(dst, src, tuples_deleted);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/* GUC assign hooks: parse string and update bit flags */
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ if (strcmp(newval, "databases") == 0)
+ evs_track_bits = EVS_TRACK_DATABASES;
+ else if (strcmp(newval, "relations") == 0)
+ evs_track_bits = EVS_TRACK_RELATIONS;
+ else
+ evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES; /* "all" or unknown */
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ if (strcmp(newval, "system") == 0)
+ evs_track_relations_bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(newval, "user") == 0)
+ evs_track_relations_bits = EVS_FILTER_USER;
+ else
+ evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER; /* "all" or unknown */
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0, NULL, evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0, NULL, evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ /* Hash of database OIDs to track specific databases */
+ evs_track_databases_hash = hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ /* Hash of (dboid, reloid) to track specific relations */
+ evs_track_relations_hash = hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+}
+
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[256];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' || !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ return;
+
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else
+ {
+ if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+ }
+ FreeFile(fp);
+}
+
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!DataDir || DataDir[0] == '\0' || !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s/%s.tmp", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(tmppath, "w");
+ if (!fp)
+ return;
+
+ fprintf(fp, "[databases]\n");
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ fprintf(fp, "%u\n", *entry);
+
+ fprintf(fp, "[relations]\n");
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ if (OidIsValid(rel_entry->dboid))
+ fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ fprintf(fp, "0 %u\n", rel_entry->reloid);
+ }
+
+ if (FreeFile(fp) != 0 || rename(tmppath, path) != 0)
+ unlink(tmppath);
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ {
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(!found); /* true if newly added */
+ }
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid, EXTVAC_OBJID(relid, type));
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid, EXTVAC_OBJID(relid, type));
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid, InvalidOid);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index def13257cbe..b8cb62d22f1 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 24b706b29ad..f8a5781bde3 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index ac66fcbdb57..b03257c6973 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 04b087e2a5c..d20d5dddcc0 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -601,6 +601,65 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
}
}
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
* Initializes the eager scan management related members of the LVRelState.
@@ -778,7 +837,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
- extvac_stats_start(rel, &extVacCounters);
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
@@ -1094,11 +1154,25 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
pgstat_progress_end_command();
@@ -3256,8 +3330,8 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
LVExtStatCountersIdx extVacCounters;
PgStat_VacuumRelationCounts extVacReport;
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3284,8 +3358,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -3314,8 +3393,8 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
LVExtStatCountersIdx extVacCounters;
PgStat_VacuumRelationCounts extVacReport;
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3341,8 +3420,13 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_cleanup_one_index(&ivinfo, istat);
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 7a85c644749..d0426e228b4 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -879,8 +879,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -909,8 +909,12 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 885d590d2b2..f0db10803d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -271,6 +271,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index c50ce51e9da..09f7775b85e 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -23,6 +23,7 @@
#include "catalog/pg_type.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
+#include "executor/instrument.h"
#include "storage/lock.h"
#include "utils/relcache.h"
#include "pgstat.h"
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7fe8e5468b8..1013a52de6e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,7 @@ typedef struct PgStat_FunctionCounts
* Working state needed to accumulate per-function-call timing statistics.
*/
/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
* Type of entry: table (heap), index, or database aggregate.
*/
typedef enum ExtVacReportType
@@ -738,6 +739,16 @@ extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v29-0001-Introduce-new-statistics-tracking-the-number-of-time.patch (9.3K, 2-v29-0001-Introduce-new-statistics-tracking-the-number-of-time.patch)
download | inline diff:
From 7811d1b76b8e65c0eb364c8d113df7a304422a8a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 18:30:12 +0300
Subject: [PATCH 1/3] Introduce new statistics tracking the number of times the
all-visible and all-frozen bits are cleared in the visibility map
(rev_all_visible_pages and rev_all_frozen_pages). These counters, together
with the existing per-vacuum frozen page statistics (vm_new_frozen_pages,
vm_new_visible_pages), help assess how aggressively vacuum is configured and
how frequently the backend has to revoke all-frozen/all-visible bits due to
concurrent modifications.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/visibilitymap.c | 10 ++++++++++
src/backend/catalog/system_views.sql | 4 +++-
src/backend/utils/activity/pgstat_relation.c | 2 ++
src/backend/utils/adt/pgstatfuncs.c | 6 ++++++
src/include/catalog/pg_proc.dat | 12 +++++++++++-
src/include/pgstat.h | 18 +++++++++++++++++-
src/test/regress/expected/rules.out | 12 +++++++++---
7 files changed, 58 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..9ea7a068ef0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ecb7c996e86..1242eca7304 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -741,7 +741,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..885d590d2b2 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 50ea9e8fb83..83ff1fff87d 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..252eab079d6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12831,6 +12831,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..02fbb8480dd 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,9 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -466,6 +469,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autoanalyze_time;
TimestampTz stat_reset_time;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_StatTabEntry;
/* ------
@@ -725,6 +730,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index deb6e2ad6a9..e392428377d 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2279,7 +2281,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2334,7 +2338,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
[text/plain] v29-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch (17.9K, 3-v29-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch)
download | inline diff:
From f78253895ef7e489e59389d60bc596b7cf42a19e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
Vacuum statistics are stored separately from regular relation and
database statistics. Dedicated PGSTAT_KIND_VACUUM_RELATION and
PGSTAT_KIND_VACUUM_DB entries in the cumulative statistics system
allocate memory for vacuum-specific metrics. Statistics are gathered
separately for tables and indexes according to vacuum phases. The
ExtVacReport union and type field distinguish PGSTAT_EXTVAC_TABLE vs
PGSTAT_EXTVAC_INDEX. Heap vacuum stats are sent to the cumulative
statistics system after vacuum has processed the indexes. Database
vacuum statistics aggregate per-table and per-index statistics within
the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 140 ++++++++++++++++++++++++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 8 ++
src/include/commands/vacuum.h | 28 ++++++
src/include/pgstat.h | 58 +++++++++++
5 files changed, 238 insertions(+)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 82c5b28e0ad..04b087e2a5c 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -282,6 +282,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -402,6 +404,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -488,6 +499,107 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -646,7 +758,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params.log_vacuum_min_duration >= 0));
@@ -663,6 +778,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -690,7 +807,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -801,6 +920,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -977,6 +1099,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
starttime);
+
pgstat_progress_end_command();
if (instrument)
@@ -3018,6 +3141,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3129,6 +3253,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3147,6 +3275,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3155,6 +3284,9 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3179,6 +3311,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3198,12 +3334,16 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 62c1ebdfd9b..faeab06d2bc 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2541,6 +2544,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 279108ca89f..7a85c644749 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,6 +869,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -877,6 +879,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -905,6 +909,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1055,6 +1062,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index e885a4b9c77..c50ce51e9da 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -333,6 +334,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 02fbb8480dd..7fe8e5468b8 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -92,6 +92,63 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -680,6 +737,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
[text/plain] v29-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch (143.0K, 4-v29-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch)
download | inline diff:
From 7c42b68e7ebeeceb1475502d2ab5e2f9bd543670 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 3 Mar 2026 00:17:13 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics extension for extended vacuum
statistics.
This commit introduces the ext_vacuum_statistics extension, which provides
extended vacuum statistics through a dedicated schema and views. Statistics
are stored via the pgstat custom statistics infrastructure. The extension
registers set_report_vacuum_hook to receive vacuum metrics and persists
them into custom stats; when the hook is not set, no additional overhead
is incurred.
Views pg_stats_vacuum_tables, pg_stats_vacuum_indexes and
pg_stats_vacuum_database expose per-table, per-index and aggregated
per-database vacuum statistics respectively.
GUCs control which objects are tracked and how. vacuum_statistics.enabled
(default on) turns collection on or off. vacuum_statistics.object_types
(default all) restricts tracking to databases only, relations only, or both.
When tracking relations, vacuum_statistics.track_relations (default all)
filters by system or user tables. vacuum_statistics.track_databases_from_list
and vacuum_statistics.track_relations_from_list (both default off) restrict
tracking to databases and relations explicitly added via add_track_database
and add_track_relation; when off, all objects of the chosen types are tracked.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 +++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 260 +++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 +++++
.../t/054_vacuum_extending_gucs_test.pl | 203 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1000 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 +++++++++
doc/src/sgml/filelist.sgml | 1 +
src/backend/access/heap/vacuumlazy.c | 112 +-
src/backend/commands/vacuumparallel.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 24 +
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 11 +
23 files changed, 3576 insertions(+), 18 deletions(-)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 2f0a88d3f77..6e064c566aa 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..4f0b1877f90
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,260 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT VOLATILE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT VOLATILE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT VOLATILE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..0ba52f7988f
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_rev_all_frozen_pages(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_rev_all_visible_pages(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..b8d5bf30ecf
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,203 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra));
+ $node->safe_psql($db, $sql);
+ sleep(0.1);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no stats when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..1f6f3e90614
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1000 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+
+/* Hash tables for track_databases and track_relations_list */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+static void evs_track_load_file(void);
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+#define ACCUM_IF(dst, src, field) \
+ do { (dst)->field += (src)->field; } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read);
+ ACCUM_IF(dst, src, total_blks_hit);
+ ACCUM_IF(dst, src, total_blks_dirtied);
+ ACCUM_IF(dst, src, total_blks_written);
+ ACCUM_IF(dst, src, blks_fetched);
+ ACCUM_IF(dst, src, blks_hit);
+ ACCUM_IF(dst, src, blk_read_time);
+ ACCUM_IF(dst, src, blk_write_time);
+ ACCUM_IF(dst, src, delay_time);
+ ACCUM_IF(dst, src, total_time);
+ ACCUM_IF(dst, src, wal_records);
+ ACCUM_IF(dst, src, wal_fpi);
+ ACCUM_IF(dst, src, wal_bytes);
+ ACCUM_IF(dst, src, wraparound_failsafe_count);
+ ACCUM_IF(dst, src, interrupts_count);
+ ACCUM_IF(dst, src, tuples_deleted);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/* GUC assign hooks: parse string and update bit flags */
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ if (strcmp(newval, "databases") == 0)
+ evs_track_bits = EVS_TRACK_DATABASES;
+ else if (strcmp(newval, "relations") == 0)
+ evs_track_bits = EVS_TRACK_RELATIONS;
+ else
+ evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES; /* "all" or unknown */
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ if (strcmp(newval, "system") == 0)
+ evs_track_relations_bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(newval, "user") == 0)
+ evs_track_relations_bits = EVS_FILTER_USER;
+ else
+ evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER; /* "all" or unknown */
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0, NULL, evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0, NULL, evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ /* Hash of database OIDs to track specific databases */
+ evs_track_databases_hash = hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ /* Hash of (dboid, reloid) to track specific relations */
+ evs_track_relations_hash = hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+}
+
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[256];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' || !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ return;
+
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else
+ {
+ if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+ }
+ FreeFile(fp);
+}
+
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!DataDir || DataDir[0] == '\0' || !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s/%s.tmp", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(tmppath, "w");
+ if (!fp)
+ return;
+
+ fprintf(fp, "[databases]\n");
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ fprintf(fp, "%u\n", *entry);
+
+ fprintf(fp, "[relations]\n");
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ if (OidIsValid(rel_entry->dboid))
+ fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ fprintf(fp, "0 %u\n", rel_entry->reloid);
+ }
+
+ if (FreeFile(fp) != 0 || rename(tmppath, path) != 0)
+ unlink(tmppath);
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ {
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(!found); /* true if newly added */
+ }
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid, EXTVAC_OBJID(relid, type));
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid, EXTVAC_OBJID(relid, type));
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid, InvalidOid);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index def13257cbe..b8cb62d22f1 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 24b706b29ad..f8a5781bde3 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index ac66fcbdb57..b03257c6973 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 04b087e2a5c..d20d5dddcc0 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -601,6 +601,65 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
}
}
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
* Initializes the eager scan management related members of the LVRelState.
@@ -778,7 +837,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
- extvac_stats_start(rel, &extVacCounters);
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
@@ -1094,11 +1154,25 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
pgstat_progress_end_command();
@@ -3256,8 +3330,8 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
LVExtStatCountersIdx extVacCounters;
PgStat_VacuumRelationCounts extVacReport;
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3284,8 +3358,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -3314,8 +3393,8 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
LVExtStatCountersIdx extVacCounters;
PgStat_VacuumRelationCounts extVacReport;
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3341,8 +3420,13 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_cleanup_one_index(&ivinfo, istat);
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 7a85c644749..d0426e228b4 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -879,8 +879,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -909,8 +909,12 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 885d590d2b2..f0db10803d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -271,6 +271,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index c50ce51e9da..09f7775b85e 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -23,6 +23,7 @@
#include "catalog/pg_type.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
+#include "executor/instrument.h"
#include "storage/lock.h"
#include "utils/relcache.h"
#include "pgstat.h"
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7fe8e5468b8..1013a52de6e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,7 @@ typedef struct PgStat_FunctionCounts
* Working state needed to accumulate per-function-call timing statistics.
*/
/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
* Type of entry: table (heap), index, or database aggregate.
*/
typedef enum ExtVacReportType
@@ -738,6 +739,16 @@ extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-12 12:02 Andrei Lepikhov <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Andrei Lepikhov @ 2026-03-12 12:02 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 9/3/26 16:46, Alena Rybakina wrote:
> I discovered that my last patches were incorrectly formed. I updated the
> correct version.
I see that v29-0001-* is a quite separate feature itself at the moment.
It makes sense to remove the commit message phrase for
vm_new_frozen_pages and vm_new_visible_pages, introduced in later patches.
This patch itself looks good to me.
--
regards, Andrei Lepikhov,
pgEdge
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-12 13:28 Andrei Lepikhov <[email protected]>
parent: Andrei Lepikhov <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Andrei Lepikhov @ 2026-03-12 13:28 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 12/3/26 13:02, Andrei Lepikhov wrote:
> On 9/3/26 16:46, Alena Rybakina wrote:
>> I discovered that my last patches were incorrectly formed. I updated
>> the correct version.
>
> I see that v29-0001-* is a quite separate feature itself at the moment.
> It makes sense to remove the commit message phrase for
> vm_new_frozen_pages and vm_new_visible_pages, introduced in later patches.
> This patch itself looks good to me.
Since this patch is almost ready for commit, I reviewed it carefully. I
noticed a documentation entry was missing, so I added it. Please see the
attachment.
While updating the patch file, I also made a few small adjustments,
including changing the parameter order in the struct and VIEW. The
commit message is also fixed.
In addition, it makes sense to discuss how these parameters are supposed
to be used. I see the following use cases:
1. Which tables have the most VM churn? - monitoring
rev_all_visible_pages normalised on the table size and its average tuple
width might expose the most suspicious tables (in terms of table
statistics).
2. DML Skew. Dividing rev_all_visible_pages by the number of tuple
updates/deletes, normalised by the average table and tuple sizes, might
indicate whether changes are localised within the table.
3. IndexOnlyScan effectiveness. Considering the speed of
rev_all_visible_pages change, normalised to the value of the
relallvisible statistic, we may detect tables where Index-Only Scan
might be inefficiently used.
Feel free to criticise it or add your own - I’m just a developer, not a
DBA. Also, I’m not sure what use cases there are for the
rev_all_frozen_pages parameter.
--
regards, Andrei Lepikhov,
pgEdge
From 96789144424e991aab44e7c8dfad9db4a2e368e1 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 18:30:12 +0300
Subject: [PATCH v30] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 ++++++++++++++++++++
src/backend/access/heap/visibilitymap.c | 10 ++++++
src/backend/catalog/system_views.sql | 2 ++
src/backend/utils/activity/pgstat_relation.c | 2 ++
src/backend/utils/adt/pgstatfuncs.c | 6 ++++
src/include/catalog/pg_proc.dat | 12 +++++++-
src/include/pgstat.h | 17 ++++++++++-
src/test/regress/expected/rules.out | 6 ++++
8 files changed, 85 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index cc014564c97..8ce0d0dd2cb 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4249,6 +4249,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-visible bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-visible, for example during an <command>INSERT</command>,
+ <command>UPDATE</command>, or <command>DELETE</command>.
+ A high rate of change in this counter means that index-only scans
+ on this table may frequently need to fall back to heap fetches,
+ and that vacuum must re-do visibility map work on those pages.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-frozen bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-frozen. A high value compared to the number of vacuum cycles
+ indicates that DML activity is frequently undoing the freezing work
+ performed by vacuum.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..9ea7a068ef0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 339c016e510..1eaf79fdb4e 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -729,6 +729,8 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_dead_tuples(C.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(C.oid) as last_vacuum,
pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum,
pg_stat_get_last_analyze_time(C.oid) as last_analyze,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..bb26e97898d 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5ac022274a7..6d7c4cc1ed2 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..252eab079d6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12831,6 +12831,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..849eea24f29 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +219,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -450,6 +452,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -725,6 +729,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f373ad704b6..4fb3167e99c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1834,6 +1834,8 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_dead_tuples(c.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum,
pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum,
pg_stat_get_last_analyze_time(c.oid) AS last_analyze,
@@ -2285,6 +2287,8 @@ pg_stat_sys_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
@@ -2340,6 +2344,8 @@ pg_stat_user_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
--
2.53.0
Attachments:
[text/plain] v30-0001-Track-table-VM-stability.patch (11.0K, 2-v30-0001-Track-table-VM-stability.patch)
download | inline diff:
From 96789144424e991aab44e7c8dfad9db4a2e368e1 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 18:30:12 +0300
Subject: [PATCH v30] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 ++++++++++++++++++++
src/backend/access/heap/visibilitymap.c | 10 ++++++
src/backend/catalog/system_views.sql | 2 ++
src/backend/utils/activity/pgstat_relation.c | 2 ++
src/backend/utils/adt/pgstatfuncs.c | 6 ++++
src/include/catalog/pg_proc.dat | 12 +++++++-
src/include/pgstat.h | 17 ++++++++++-
src/test/regress/expected/rules.out | 6 ++++
8 files changed, 85 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index cc014564c97..8ce0d0dd2cb 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4249,6 +4249,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-visible bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-visible, for example during an <command>INSERT</command>,
+ <command>UPDATE</command>, or <command>DELETE</command>.
+ A high rate of change in this counter means that index-only scans
+ on this table may frequently need to fall back to heap fetches,
+ and that vacuum must re-do visibility map work on those pages.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-frozen bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-frozen. A high value compared to the number of vacuum cycles
+ indicates that DML activity is frequently undoing the freezing work
+ performed by vacuum.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..9ea7a068ef0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 339c016e510..1eaf79fdb4e 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -729,6 +729,8 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_dead_tuples(C.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(C.oid) as last_vacuum,
pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum,
pg_stat_get_last_analyze_time(C.oid) as last_analyze,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..bb26e97898d 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5ac022274a7..6d7c4cc1ed2 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..252eab079d6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12831,6 +12831,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..849eea24f29 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +219,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -450,6 +452,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -725,6 +729,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f373ad704b6..4fb3167e99c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1834,6 +1834,8 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_dead_tuples(c.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum,
pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum,
pg_stat_get_last_analyze_time(c.oid) AS last_analyze,
@@ -2285,6 +2287,8 @@ pg_stat_sys_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
@@ -2340,6 +2344,8 @@ pg_stat_user_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
--
2.53.0
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-12 18:10 Alena Rybakina <[email protected]>
parent: Andrei Lepikhov <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2026-03-12 18:10 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 12.03.2026 18:28, Andrei Lepikhov wrote:
>
> In addition, it makes sense to discuss how these parameters are
> supposed to be used. I see the following use cases:
>
> 1. Which tables have the most VM churn? - monitoring
> rev_all_visible_pages normalised on the table size and its average
> tuple width might expose the most suspicious tables (in terms of table
> statistics).
> 2. DML Skew. Dividing rev_all_visible_pages by the number of tuple
> updates/deletes, normalised by the average table and tuple sizes,
> might indicate whether changes are localised within the table.
> 3. IndexOnlyScan effectiveness. Considering the speed of
> rev_all_visible_pages change, normalised to the value of the
> relallvisible statistic, we may detect tables where Index-Only Scan
> might be inefficiently used.
>
>
I agree with all these points and I think we can add it in the
documentation.
On 12.03.2026 17:02, Andrei Lepikhov wrote:
> On 9/3/26 16:46, Alena Rybakina wrote:
>> I discovered that my last patches were incorrectly formed. I updated
>> the correct version.
>
> I see that v29-0001-* is a quite separate feature itself at the
> moment. It makes sense to remove the commit message phrase for
> vm_new_frozen_pages and vm_new_visible_pages, introduced in later
> patches.
> This patch itself looks good to me.
BTW, I have noticed that my third patch (from 29th - when I have added
ext_vacuum_statistics) is huge but I have no idea how to split it
logically. I'm not sure that separation by objects can simplify the
review process. Maybe I should add only base logic for the extension and
then gucs, what do you think?
Any suggestions are welcome here.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-13 13:04 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 2 replies; 77+ messages in thread
From: Alena Rybakina @ 2026-03-13 13:04 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 13.03.2026 15:51, Alena Rybakina wrote:
>>>
>>> In addition, it makes sense to discuss how these parameters are
>>> supposed to be used. I see the following use cases:
>>>
>>> 1. Which tables have the most VM churn? - monitoring
>>> rev_all_visible_pages normalised on the table size and its average
>>> tuple width might expose the most suspicious tables (in terms of
>>> table statistics).
>>> 2. DML Skew. Dividing rev_all_visible_pages by the number of tuple
>>> updates/deletes, normalised by the average table and tuple sizes,
>>> might indicate whether changes are localised within the table.
>>> 3. IndexOnlyScan effectiveness. Considering the speed of
>>> rev_all_visible_pages change, normalised to the value of the
>>> relallvisible statistic, we may detect tables where Index-Only Scan
>>> might be inefficiently used.
>>
>> With the parameter that was included before (pg_class_relallfrozen
>> and relallvisible
>> https://github.com/MasaoFujii/postgresql/commit/99f8f3fbbc8f743290844e8c676d39dad11c5d5d)
>> in the pg_stat_tables, I think I can provide isolation test to prove
>> it - I can use my isolation test
>> vacuum-extending-in-repetable-read.spec that I have added in the
>> extension (ext_vacuum_statistics). What do you think?
>
> I've prepared the test. Do you think it would make sense to include it
> in 0001?
>
I have added it in the 31th version for now and nothing else has been
changed (if you don't mind, exclude it).
From 486a29e6a22d43e2911eb849bdb3b3b39eefab91 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Fri, 13 Mar 2026 16:00:39 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 2 +
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 12 +-
src/include/pgstat.h | 17 +-
.../t/052_vacuum_extending_freeze_test.pl | 215 ++++++++++++++++++
src/test/regress/expected/rules.out | 6 +
9 files changed, 300 insertions(+), 2 deletions(-)
create mode 100644 src/test/recovery/t/052_vacuum_extending_freeze_test.pl
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index b77d189a500..fb656977b2e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4090,6 +4090,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-visible bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-visible, for example during an <command>INSERT</command>,
+ <command>UPDATE</command>, or <command>DELETE</command>.
+ A high rate of change in this counter means that index-only scans
+ on this table may frequently need to fall back to heap fetches,
+ and that vacuum must re-do visibility map work on those pages.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-frozen bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-frozen. A high value compared to the number of vacuum cycles
+ indicates that DML activity is frequently undoing the freezing work
+ performed by vacuum.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 3047bd46def..2e7c28ea307 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -161,6 +162,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 7553f31fef0..fa4c74bcd5d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -715,6 +715,8 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_dead_tuples(C.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(C.oid) as last_vacuum,
pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum,
pg_stat_get_last_analyze_time(C.oid) as last_analyze,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..bb26e97898d 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 73ca0bb0b7f..901f3dd55a1 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5e5e33f64fc..961337ce282 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12693,6 +12693,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index fff7ecc2533..04ccb3c06c2 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -156,6 +156,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -214,7 +216,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -447,6 +449,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -722,6 +726,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/recovery/t/052_vacuum_extending_freeze_test.pl b/src/test/recovery/t/052_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..384e123381f
--- /dev/null
+++ b/src/test/recovery/t/052_vacuum_extending_freeze_test.pl
@@ -0,0 +1,215 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats system using TAP
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+plan tests => 10;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('vacuum_extending_freeze_test');
+$node->init;
+
+# Configure the server for aggressive freezing behavior used by the test
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Enable necessary settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+# Polls statistics until the named columns exceed the provided
+# baseline values or until timeout.
+#
+# run_vacuum is a boolean (0 or 1) means we need to fetch frozen and visible pages
+# from pg_class table, otherwise we need to fetch frozen and visible pages from pg_stat_all_tables table.
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $run_vacuum = ($args{run_vacuum} or 0);
+ my $result_query;
+ my $sql;
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM vestat');
+
+ $sql = "
+ SELECT relallfrozen > 0
+ AND relallvisible > 0
+ FROM pg_class c
+ WHERE c.relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT rev_all_frozen_pages > 0
+ AND rev_all_visible_pages > 0
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ # sub-second sleep
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $relallvisible = 0;
+my $relallfrozen = 0;
+
+my $relallvisible_prev = 0;
+my $relallfrozen_prev = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT c.relallvisible, c.relallfrozen,
+ rev_all_visible_pages, rev_all_frozen_pages
+ FROM pg_class c
+ LEFT JOIN pg_stat_all_tables s ON s.relid = c.oid
+ WHERE c.relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($relallvisible, $relallfrozen, $rev_all_visible_pages, $rev_all_frozen_pages)
+ = split /\s+/, $base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+$node->safe_psql($dbname, q{
+ SELECT pg_stat_force_next_flush();
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 5000) AS g(x);
+ ANALYZE vestat;
+});
+
+# Poll the stats view until the expected deltas appear or timeout.
+$updated = wait_for_vacuum_stats(run_vacuum => 1);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (relallfrozen and relallvisible advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Verify initial statistics after vacuum
+#------------------------------------------------------------------------------
+ok($relallfrozen > $relallfrozen_prev, 'relallfrozen has increased');
+ok($relallvisible > $relallvisible_prev, 'relallvisible has increased');
+ok($rev_all_frozen_pages == 0, 'rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == 0, 'rev_all_visible_pages stay the same');
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+$relallfrozen_prev = $relallfrozen;
+$relallvisible_prev = $relallvisible;
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$node->safe_psql($dbname, 'SELECT pg_stat_force_next_flush()');
+
+# Poll until stats update or timeout.
+$updated = wait_for_vacuum_stats(run_vacuum => 0);
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Check updated statistics after backend activity
+#------------------------------------------------------------------------------
+
+ok($relallfrozen == $relallfrozen_prev, 'relallfrozen stay the same');
+ok($relallvisible == $relallvisible_prev, 'relallvisible stay the same');
+ok($rev_all_frozen_pages > 0, 'rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > 0, 'rev_all_visible_pages has increased');
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f4ee2bd7459..8dbf5ce34bb 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1834,6 +1834,8 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_dead_tuples(c.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum,
pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum,
pg_stat_get_last_analyze_time(c.oid) AS last_analyze,
@@ -2256,6 +2258,8 @@ pg_stat_sys_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
@@ -2311,6 +2315,8 @@ pg_stat_user_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v31-0001-Track-table-VM-stability.patch (19.2K, 2-v31-0001-Track-table-VM-stability.patch)
download | inline diff:
From 486a29e6a22d43e2911eb849bdb3b3b39eefab91 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Fri, 13 Mar 2026 16:00:39 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 2 +
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 12 +-
src/include/pgstat.h | 17 +-
.../t/052_vacuum_extending_freeze_test.pl | 215 ++++++++++++++++++
src/test/regress/expected/rules.out | 6 +
9 files changed, 300 insertions(+), 2 deletions(-)
create mode 100644 src/test/recovery/t/052_vacuum_extending_freeze_test.pl
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index b77d189a500..fb656977b2e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4090,6 +4090,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-visible bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-visible, for example during an <command>INSERT</command>,
+ <command>UPDATE</command>, or <command>DELETE</command>.
+ A high rate of change in this counter means that index-only scans
+ on this table may frequently need to fall back to heap fetches,
+ and that vacuum must re-do visibility map work on those pages.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-frozen bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-frozen. A high value compared to the number of vacuum cycles
+ indicates that DML activity is frequently undoing the freezing work
+ performed by vacuum.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 3047bd46def..2e7c28ea307 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -161,6 +162,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 7553f31fef0..fa4c74bcd5d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -715,6 +715,8 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_dead_tuples(C.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(C.oid) as last_vacuum,
pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum,
pg_stat_get_last_analyze_time(C.oid) as last_analyze,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..bb26e97898d 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 73ca0bb0b7f..901f3dd55a1 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5e5e33f64fc..961337ce282 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12693,6 +12693,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index fff7ecc2533..04ccb3c06c2 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -156,6 +156,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -214,7 +216,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -447,6 +449,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -722,6 +726,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/recovery/t/052_vacuum_extending_freeze_test.pl b/src/test/recovery/t/052_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..384e123381f
--- /dev/null
+++ b/src/test/recovery/t/052_vacuum_extending_freeze_test.pl
@@ -0,0 +1,215 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats system using TAP
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+plan tests => 10;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('vacuum_extending_freeze_test');
+$node->init;
+
+# Configure the server for aggressive freezing behavior used by the test
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Enable necessary settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+# Polls statistics until the named columns exceed the provided
+# baseline values or until timeout.
+#
+# run_vacuum is a boolean (0 or 1) means we need to fetch frozen and visible pages
+# from pg_class table, otherwise we need to fetch frozen and visible pages from pg_stat_all_tables table.
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $run_vacuum = ($args{run_vacuum} or 0);
+ my $result_query;
+ my $sql;
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM vestat');
+
+ $sql = "
+ SELECT relallfrozen > 0
+ AND relallvisible > 0
+ FROM pg_class c
+ WHERE c.relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT rev_all_frozen_pages > 0
+ AND rev_all_visible_pages > 0
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ # sub-second sleep
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $relallvisible = 0;
+my $relallfrozen = 0;
+
+my $relallvisible_prev = 0;
+my $relallfrozen_prev = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT c.relallvisible, c.relallfrozen,
+ rev_all_visible_pages, rev_all_frozen_pages
+ FROM pg_class c
+ LEFT JOIN pg_stat_all_tables s ON s.relid = c.oid
+ WHERE c.relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($relallvisible, $relallfrozen, $rev_all_visible_pages, $rev_all_frozen_pages)
+ = split /\s+/, $base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+$node->safe_psql($dbname, q{
+ SELECT pg_stat_force_next_flush();
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 5000) AS g(x);
+ ANALYZE vestat;
+});
+
+# Poll the stats view until the expected deltas appear or timeout.
+$updated = wait_for_vacuum_stats(run_vacuum => 1);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (relallfrozen and relallvisible advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Verify initial statistics after vacuum
+#------------------------------------------------------------------------------
+ok($relallfrozen > $relallfrozen_prev, 'relallfrozen has increased');
+ok($relallvisible > $relallvisible_prev, 'relallvisible has increased');
+ok($rev_all_frozen_pages == 0, 'rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == 0, 'rev_all_visible_pages stay the same');
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+$relallfrozen_prev = $relallfrozen;
+$relallvisible_prev = $relallvisible;
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$node->safe_psql($dbname, 'SELECT pg_stat_force_next_flush()');
+
+# Poll until stats update or timeout.
+$updated = wait_for_vacuum_stats(run_vacuum => 0);
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Check updated statistics after backend activity
+#------------------------------------------------------------------------------
+
+ok($relallfrozen == $relallfrozen_prev, 'relallfrozen stay the same');
+ok($relallvisible == $relallvisible_prev, 'relallvisible stay the same');
+ok($rev_all_frozen_pages > 0, 'rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > 0, 'rev_all_visible_pages has increased');
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f4ee2bd7459..8dbf5ce34bb 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1834,6 +1834,8 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_dead_tuples(c.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum,
pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum,
pg_stat_get_last_analyze_time(c.oid) AS last_analyze,
@@ -2256,6 +2258,8 @@ pg_stat_sys_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
@@ -2311,6 +2315,8 @@ pg_stat_user_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-16 08:45 Andrei Lepikhov <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Andrei Lepikhov @ 2026-03-16 08:45 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; Alena Rybakina <[email protected]>; +Cc: pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 15/3/26 18:18, Andrey Borodin wrote:
>
>
>> On 13 Mar 2026, at 18:04, Alena Rybakina <[email protected]> wrote:
>
> I've decided to take a look into v31.
>
> Overall idea of tracking VM dynamics seems good to me.
>
> But the column naming for rev_all_visible_pages and rev_all_frozen_pages
> seems strange to me. I've skimmed the thread but could not figure out what
> "rev_" stands for. Revisions? Revolutions? Reviews?
I suppose 'revert' is the exact term here. Someone decided to set the
flag, and we reverted his decision. Does this make sense to you? Anyway,
I always leave it in the natives' (and committers') hands.
>
> Is there a reason why you break "SELECT * FROM pg_stat_all_tables" for
> an existing software? IMO even if we want these columns in this exact view
> - they ought to be appended to the end of the column list.
Please specify what you mean by this 'break'?
The relational model has never guaranteed a specific order of columns
returned unless you specify their names explicitly as a list. I think it
is good if someone found a flaw in their application, depending on the
wildcard order. So, I organised the elements according to their logical
order.
What's more? If you check the history of this VIEW, you will find that
it has always been updated in logical order. Please explain your point
if I misunderstood it.
>
> Some nits about the code.
I doubt if we need a test for these parameters - they reflect the
physical structure of the storage and might be unstable. But anyway, it
should be better to live in isolation tests, as similar statistics.
Thanks for your efforts!
--
regards, Andrei Lepikhov,
pgEdge
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-16 12:07 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2026-03-16 12:07 UTC (permalink / raw)
To: Андрей Зубков <[email protected]>; Andrey Borodin <[email protected]>; +Cc: Andrei Lepikhov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
Hi! Thank you for your attention to this patch!
On 16.03.2026 11:34, Андрей Зубков wrote:
> Really it was "revocations", but I'm agree with Andrey that naming
> isn't clear. *_vm_cleared looks better, but talking about naming here
> "vm" meaning is not clear. I think it will be understood as visibility
> map, but it is "mark" really. Maybe "*_pages_marks_cleared" will be
> better?
>
> Also a macro in pgstat.h:733 and pgstat.h:738 still holds "_rev_".
Good catch, fixed.
>
> I think the docs description needs a little correction:
>
> - visible_pages_vm_cleared. I think listing of possible DML operations
> is not needed here, also it seems a high rate of this counter has no
> direct relation to the index only scans because we can have very
> agressive vacuum on a table that will do the opposite. It will hold
> few pages without visibility marks constantly but with the cost
> of high visible_pages_vm_cleared rate. My proposition follows:
>
> Number of times the all-visible bit in the
> <link linkend="storage-vm">visibility map</link> was cleared for a
> pages of this table. The all-visible bit of a heap page is cleared
> every time backend process modifies a page previously marked
> all-visible by vacuum. Vacuum process must process page once again
> on the next run. A high rate of change of this counter means that
> vacuum should re-do its work on this table.
>
> - frozen_pages_vm_cleared:
>
> Number of times the all-frozen bit in the
> <link linkend="storage-vm">visibility map</link> was cleared for a
> pages of this table. The all-frozen bit of a heap page is cleared
> every time backend process modifies a page previously marked
> all-frozen by vacuum. Vacuum process must process page once again on
> the next freeze run on this table.
I agree, this description is clearer. Fixed.
-----------
Best regards,
Alena Rybakina
From bd7cbd4450512aaf640156e977faa28e5095d33a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 16 Mar 2026 14:55:45 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 ++++++++
src/backend/access/heap/visibilitymap.c | 10 +++
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 ++
src/include/catalog/pg_proc.dat | 10 +++
src/include/pgstat.h | 17 ++++-
.../expected/vacuum-extending-freeze.out | 50 +++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 73 +++++++++++++++++++
src/test/regress/expected/rules.out | 12 ++-
11 files changed, 212 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 9c5c6dc490f..0b27558686e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4258,6 +4258,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_pages_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible bit of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by <command>VACUUM</command>. The
+ page must then be processed again by <command>VACUUM</command> on a
+ subsequent run. A high rate of change in this counter means that
+ <command>VACUUM</command> has to repeatedly re-process pages of this
+ table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_pages_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen bit of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by <command>VACUUM</command>. The page must then
+ be processed again by <command>VACUUM</command> on the next freeze
+ run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..7b3ab6244d0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_pages_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_pages_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 90d48bc9c80..9ff013ac797 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -741,7 +741,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_pages_cleared(C.oid) AS visible_pages_cleared,
+ pg_stat_get_frozen_pages_cleared(C.oid) AS frozen_pages_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..78936aca82e 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_pages_cleared += lstats->counts.visible_pages_cleared;
+ tabentry->frozen_pages_cleared += lstats->counts.frozen_pages_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5ac022274a7..d50b7233c0e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_pages_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_pages_cleared)
+
+/* pg_stat_get_frozen_pages_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_pages_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..b52e463e63f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12833,4 +12833,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_pages_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_pages_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_pages_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_pages_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..8116d0959de 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_pages_cleared;
+ PgStat_Counter frozen_pages_cleared;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +219,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -450,6 +452,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_pages_cleared;
+ PgStat_Counter frozen_pages_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -725,6 +729,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_visible_pages_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_pages_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_pages_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_pages_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..58b51570e5e
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,50 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1_initial_vacuum s2_vacuum s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_initial_vacuum:
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum:
+ VACUUM vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT relallfrozen > 0 AS relallfrozen_pos,
+ relallvisible > 0 AS relallvisible_pos
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+relallfrozen_pos|relallvisible_pos
+----------------+-----------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT visible_pages_cleared > 0 AS visible_pages_cleared,
+ frozen_pages_cleared > 0 AS frozen_pages_cleared
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat';
+
+visible_pages_cleared|frozen_pages_cleared
+---------------------+--------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 4e466580cd4..81e68f85d88 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -124,3 +124,4 @@ test: serializable-parallel-2
test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..b8f8c177595
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,73 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+
+setup
+{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i FROM generate_series(1, 5000) AS g(i);
+
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_initial_vacuum
+{
+ SELECT pg_stat_force_next_flush();
+}
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT relallfrozen > 0 AS relallfrozen_pos,
+ relallvisible > 0 AS relallvisible_pos
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT visible_pages_cleared > 0 AS visible_pages_cleared,
+ frozen_pages_cleared > 0 AS frozen_pages_cleared
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat';
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_vacuum
+{
+ VACUUM vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s1_initial_vacuum
+ s2_vacuum
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 71d7262049e..b36b551d877 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_pages_cleared(c.oid) AS visible_pages_cleared,
+ pg_stat_get_frozen_pages_cleared(c.oid) AS frozen_pages_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2298,7 +2300,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_pages_cleared,
+ frozen_pages_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2353,7 +2357,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_pages_cleared,
+ frozen_pages_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v34-0001-Track-table-VM-stability.patch (15.5K, 2-v34-0001-Track-table-VM-stability.patch)
download | inline diff:
From bd7cbd4450512aaf640156e977faa28e5095d33a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 16 Mar 2026 14:55:45 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 ++++++++
src/backend/access/heap/visibilitymap.c | 10 +++
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 ++
src/include/catalog/pg_proc.dat | 10 +++
src/include/pgstat.h | 17 ++++-
.../expected/vacuum-extending-freeze.out | 50 +++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 73 +++++++++++++++++++
src/test/regress/expected/rules.out | 12 ++-
11 files changed, 212 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 9c5c6dc490f..0b27558686e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4258,6 +4258,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_pages_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible bit of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by <command>VACUUM</command>. The
+ page must then be processed again by <command>VACUUM</command> on a
+ subsequent run. A high rate of change in this counter means that
+ <command>VACUUM</command> has to repeatedly re-process pages of this
+ table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_pages_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen bit of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by <command>VACUUM</command>. The page must then
+ be processed again by <command>VACUUM</command> on the next freeze
+ run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..7b3ab6244d0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_pages_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_pages_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 90d48bc9c80..9ff013ac797 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -741,7 +741,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_pages_cleared(C.oid) AS visible_pages_cleared,
+ pg_stat_get_frozen_pages_cleared(C.oid) AS frozen_pages_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..78936aca82e 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_pages_cleared += lstats->counts.visible_pages_cleared;
+ tabentry->frozen_pages_cleared += lstats->counts.frozen_pages_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5ac022274a7..d50b7233c0e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_pages_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_pages_cleared)
+
+/* pg_stat_get_frozen_pages_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_pages_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..b52e463e63f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12833,4 +12833,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_pages_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_pages_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_pages_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_pages_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..8116d0959de 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_pages_cleared;
+ PgStat_Counter frozen_pages_cleared;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +219,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -450,6 +452,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_pages_cleared;
+ PgStat_Counter frozen_pages_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -725,6 +729,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_visible_pages_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_pages_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_pages_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_pages_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..58b51570e5e
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,50 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1_initial_vacuum s2_vacuum s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_initial_vacuum:
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum:
+ VACUUM vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT relallfrozen > 0 AS relallfrozen_pos,
+ relallvisible > 0 AS relallvisible_pos
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+relallfrozen_pos|relallvisible_pos
+----------------+-----------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT visible_pages_cleared > 0 AS visible_pages_cleared,
+ frozen_pages_cleared > 0 AS frozen_pages_cleared
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat';
+
+visible_pages_cleared|frozen_pages_cleared
+---------------------+--------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 4e466580cd4..81e68f85d88 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -124,3 +124,4 @@ test: serializable-parallel-2
test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..b8f8c177595
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,73 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+
+setup
+{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i FROM generate_series(1, 5000) AS g(i);
+
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_initial_vacuum
+{
+ SELECT pg_stat_force_next_flush();
+}
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT relallfrozen > 0 AS relallfrozen_pos,
+ relallvisible > 0 AS relallvisible_pos
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT visible_pages_cleared > 0 AS visible_pages_cleared,
+ frozen_pages_cleared > 0 AS frozen_pages_cleared
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat';
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_vacuum
+{
+ VACUUM vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s1_initial_vacuum
+ s2_vacuum
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 71d7262049e..b36b551d877 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_pages_cleared(c.oid) AS visible_pages_cleared,
+ pg_stat_get_frozen_pages_cleared(c.oid) AS frozen_pages_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2298,7 +2300,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_pages_cleared,
+ frozen_pages_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2353,7 +2357,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_pages_cleared,
+ frozen_pages_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-16 12:11 Alena Rybakina <[email protected]>
parent: Andrei Lepikhov <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2026-03-16 12:11 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; Andrey Borodin <[email protected]>; +Cc: pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 16.03.2026 11:45, Andrei Lepikhov wrote:
> On 15/3/26 18:18, Andrey Borodin wrote:
>>> On 13 Mar 2026, at 18:04, Alena Rybakina <[email protected]>
>>> wrote:
>>
>> I've decided to take a look into v31.
>>
>> Overall idea of tracking VM dynamics seems good to me.
>>
>> But the column naming for rev_all_visible_pages and rev_all_frozen_pages
>> seems strange to me. I've skimmed the thread but could not figure out
>> what
>> "rev_" stands for. Revisions? Revolutions? Reviews?
>
> I suppose 'revert' is the exact term here. Someone decided to set the
> flag, and we reverted his decision. Does this make sense to you?
> Anyway, I always leave it in the natives' (and committers') hands.
I think renaming them to 'cleared' helps avoid the confusion.
I have adopted the names proposed by A. Zubkov in v34.
>> Some nits about the code.
>
> I doubt if we need a test for these parameters - they reflect the
> physical structure of the storage and might be unstable. But anyway,
> it should be better to live in isolation tests, as similar statistics.
>
I moved the tests there. Regression tests are unfortunately not an
option because the statistics are not stable.
If the isolation test turns out to be unstable again, I'll move them
back to the TAP tests as I initially implemented,
following A. Borodin's suggestion.
See the version in the
https://www.postgresql.org/message-id/767d28c9-2ae8-43df-9f2e-3e8785075115%40yandex.ru
--
-----------
Best regards,
Alena Rybakina
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-16 14:27 Andrey Borodin <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Andrey Borodin @ 2026-03-16 14:27 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Andrei Lepikhov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
> On 16 Mar 2026, at 17:11, Alena Rybakina <[email protected]> wrote:
>
> I moved the tests there. Regression tests are unfortunately not an option because the statistics are not stable.
I think there's no need to test for correct numbers. It would be totally enough
to just for sane numbers or even any numbers at all.
If the test just invokes the function without segfaulting - that's already by far
better than no test at all.
Of course, test that verifies expected behavior is better, if it's possible.
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-17 15:27 Andrei Zubkov <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Andrei Zubkov @ 2026-03-17 15:27 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Andrey Borodin <[email protected]>; Andrei Lepikhov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
Alena Rybakina <[email protected]> writes:
Hi, Alena!
I have some thoughts about your descriptions in the docs (V34). It looks
clearer now, but in seems to me it still contains some inaccuracy..
1. Naming: The meaning of 'visible/frozen_pages_cleared' seems to me as
cleared pages rather then cleared marks... Maybe it should be
'visible_page_marks_cleared'?
2. Mention of a <command>VACUUM</command> in the docs may be understood
as related to manual VACUUM command only. However, autovacuum is
accounted as well.. I think we can use just term 'vacuum' here as a
facility rather than command.
--
regards, Andrei Zubkov
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-18 14:19 Andrei Zubkov <[email protected]>
parent: Andrei Zubkov <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Andrei Zubkov @ 2026-03-18 14:19 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Andrey Borodin <[email protected]>; Andrei Lepikhov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
Alena Rybakina <[email protected]> writes:
> 1. Naming: The meaning of 'visible/frozen_pages_cleared' seems to me as
> cleared pages rather then cleared marks... Maybe it should be
> 'visible_page_marks_cleared'?
>
> I agree, this improves clarity. I've renamed it as proposed.
>
> 2. Mention of a <command>VACUUM</command> in the docs may be understood
> as related to manual VACUUM command only. However, autovacuum is
> accounted as well.. I think we can use just term 'vacuum' here as a
> facility rather than command.
>
> Fixed. Clarified that this applies to both manual
> <command>VACUUM</command> and autovacuum.
I think it is good now
>
> I also added an additional test scenario where one process holds a
> transaction open while another process deletes tuples. We expect that
> the all-visible and all-frozen flags, previously set by VACUUM, are
> cleared only after the deleting transaction commits and the changes
> become visible.
>
I'm happy with the patch now.
--
Best regards, Andrei Zubkov
Postgres Professional
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-03-30 06:13 Alena Rybakina <[email protected]>
parent: Andrey Borodin <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2026-03-30 06:13 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; Andrey Borodin <[email protected]>; Andrei Lepikhov <[email protected]>; Andrei Zubkov <[email protected]>; +Cc: pgsql-hackers; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
Hi, all!
On 17.03.2026 21:11, Alena Rybakina wrote:
> I think last version is stable - it is in the isolation test. The last
> version is here
> https://www.postgresql.org/message-id/68939c47-fa0c-4198-853a-92d1390079da%40yandex.ru
>
Nothing special has been changed. I have rebased the patch because of
updated PGSTAT_FILE_FORMAT_ID.
-----------
Best regards,
Alena Rybakina
From 7cea0dfa3c30805797a0a3d6ca8f8ac9b617d4a8 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index bb75ed1069b..83c9e265624 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4362,6 +4362,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index e54018004db..855ea965583 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..78c0e6329bc 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 9185a8e6b83..2e2b6897d36 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0118e970dda..f6028006776 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12851,4 +12851,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 8e3549c3752..3a6d75892fa 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -753,6 +757,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 4e466580cd4..81e68f85d88 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -124,3 +124,4 @@ test: serializable-parallel-2
test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2b3cf6d8569..9036eb29988 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2304,7 +2306,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2359,7 +2363,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v36-0001-Track-table-VM-stability.patch (21.7K, 2-v36-0001-Track-table-VM-stability.patch)
download | inline diff:
From 7cea0dfa3c30805797a0a3d6ca8f8ac9b617d4a8 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index bb75ed1069b..83c9e265624 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4362,6 +4362,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index e54018004db..855ea965583 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..78c0e6329bc 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 9185a8e6b83..2e2b6897d36 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0118e970dda..f6028006776 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12851,4 +12851,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 8e3549c3752..3a6d75892fa 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -753,6 +757,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 4e466580cd4..81e68f85d88 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -124,3 +124,4 @@ test: serializable-parallel-2
test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2b3cf6d8569..9036eb29988 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2304,7 +2306,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2359,7 +2363,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-04-14 11:10 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2026-04-14 11:10 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrey Borodin <[email protected]>; Andrei Zubkov <[email protected]>; Andrei Lepikhov <[email protected]>
On 30.03.2026 09:13, Alena Rybakina wrote:
> Hi, all!
>
> On 17.03.2026 21:11, Alena Rybakina wrote:
>
>> I think last version is stable - it is in the isolation test. The
>> last version is here
>> https://www.postgresql.org/message-id/68939c47-fa0c-4198-853a-92d1390079da%40yandex.ru
>>
> Nothing special has been changed. I have rebased the patch because of
> updated PGSTAT_FILE_FORMAT_ID.
>
I have rebased the patch.
--
-----------
Best regards,
Alena Rybakina
From 5d8eddaf00d2729c3bc20dfb20debff1833b240e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v37-0001-Track-table-VM-stability.patch (21.7K, 2-v37-0001-Track-table-VM-stability.patch)
download | inline diff:
From 5d8eddaf00d2729c3bc20dfb20debff1833b240e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-04-28 02:16 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 77+ messages in thread
From: Alena Rybakina @ 2026-04-28 02:16 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrey Borodin <[email protected]>; Andrei Zubkov <[email protected]>; Andrei Lepikhov <[email protected]>
Hi, all!
I have updated the core patch that implements the machinery for
collecting extended vacuum statistics (I didn't touch the first patch
that is ready for commit, only patches that are related to extension),
and rebased the ext_vacuum_statistics extension on top of it. The split
is intentional: the core only gathers metrics and hands them out, while
the actual storage and SQL-level access to the statistics live entirely
in the extension. If the extension is not loaded, the overhead is
essentially zero - we only fill a small struct on the stack and do a
NULL check on the hook.
What was updated in the core
The core gains the machinery and the hook through which the extension
receives metrics after each vacuum.
The hook. A new hook has been added in pgstat - set_report_vacuum_hook.
It is fired once per vacuumed table and once per vacuumed index, plus
when forming the per-database aggregate. The extension registers its
handler in _PG_init and by default the hook is NULL, so without an
extension the core behaves exactly as before.
The set of statistics is the same as before. Common to tables, indexes
and the database - hits and misses in shared buffers, number of dirtied
and written pages, WAL volume, buffer read and write times, sleep time
spent in delay points, total wall-clock vacuum time (including I/O and
lock waits), counter of emergency anti-wraparound vacuums, number of
interrupts and removed tuples. Tables additionally report frozen tuples,
pages marked all-frozen / all-visible in the visibility map, number of
scanned and removed pages, number of index passes, etc. Indexes report
freed pages.
The least obvious part of the implementation is subtracting index
statistics from the table statistics. This is the bit worth
highlighting. The thing is that indexes are vacuumed before the heap,
and the buffer and WAL statistics that we capture at the heap level by
the end of the heap vacuum already include everything that was spent on
the indexes. If we simply expose the diff of pgBufferUsage/pgWalUsage
between start and end, the table ends up with double-counted pages/WAL:
once in its own report, and a second time inside the reports of its
indexes. This is especially noticeable with parallel index vacuum:
workers accumulate their usage in the leader only after they finish, so
without subtraction the heap report would receive the combined cost of
all workers as a "bonus".
To handle this, as each index finishes vacuuming, its counters are
accumulated into the state of the current operation, and at the moment
the heap report is built these sums are subtracted out. As a result, the
extension receives clean numbers: "this is what was actually spent on
the table itself", and separately "this is what was actually spent on
each index". The behaviour is idempotent for both serial and parallel
vacuum.
The ext_vacuum_statistics extension
The extension registers the hook handler and stores the received data
through the pgstat custom statistics infrastructure. That is, vacuum
counters are kept not in the extension's own files, but together with
the regular cumulative statistics - they survive a restart and are reset
together with pg_stat_reset_*. Access is provided through three views:
one for tables, one for indexes, and one with the per-database aggregate.
Filtering
This is where the main flexibility lives - the extension does not force
"collect everything", but lets you choose both what to track and which
metrics to keep.
By object type. You can limit collection to databases only (without
per-table detail), to tables only, or collect both. Among tables, you
can additionally filter system / user / all.
By an explicit list. An alternative to "by type" is a whitelist: you
turn the corresponding mode on, and the extension starts collecting
statistics only for the databases and tables that were explicitly
registered via add_track_database / add_track_relation (with matching
remove_* for removal). When the lists are off, the type filter is in
effect; when they are on, only the list applies. This is convenient when
you are interested in monitoring specific "hot" tables and do not want
to spend memory on statistics for everything else.
This list is persisted to disk, and there is one more non-trivial part
here. List changes are concurrent - multiple sessions may call
add_track_* simultaneously, plus there is an object-access hook that
cleans the entry on DROP. To avoid ending up with a torn file, access to
the list is serialized via a dedicated LWLock tranche (requested from a
shmem_request_hook), and the file itself is written atomically: first
into a temporary file, then fflush + pg_fsync + durable_rename. All I/O
return codes are checked; on error the temporary file is removed and the
real one is left untouched; PG_TRY/PG_CATCH guarantees cleanup on
ereport(ERROR). Reading the list takes the same lock in shared mode, so
a concurrent write cannot tear the load.
By metric category. There is also a GUC that takes a list and turns on
the categories of interest - buffers, WAL, general counters, timings (or
all). Unwanted categories are simply skipped on the hook handler side
and never make it into the pgstat entry, which reduces the overhead of
the handler itself. This is useful when, for example, only timings are
needed - in that case the extension does not waste time copying the
buffer and WAL fields.
Privileges. The add_track_* / remove_track_* functions require superuser
or pg_read_all_stats. At the SQL level, EXECUTE is revoked from PUBLIC
and granted only to pg_read_all_stats, so a regular user has no access
to mutating the list. The views are unrestricted, like regular statistics.
What is in the patches
0002-Machinery-for-grabbing-extended-vacuum-statistics.patch - the
machinery in the core plus the hook.
0003-ext_vacuum_statistics-...patch - the extension itself, filtering,
views, tests.
-----------
Best regards,
Alena Rybakina
Yandex Cloud
From 19f5a39f7e97d3fc2d18415ba2c51ffcd3b32f49 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH 1/3] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
From 3a5e0bd82578d1fea63d6bda229dc4d0b224684e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add infrastructure inside lazy vacuum to gather extended per-vacuum
metrics and expose them to extensions via a new hook. Core itself
does not persist these metrics — that is the job of an extension
(see ext_vacuum_statistics).
Statistics are gathered separately for tables and indexes according
to vacuum phases. The ExtVacReport union and type field distinguish
PGSTAT_EXTVAC_TABLE vs PGSTAT_EXTVAC_INDEX. Heap vacuum stats are
sent to the cumulative statistics system after vacuum has processed
the indexes. Database vacuum statistics aggregate per-table and
per-index statistics within the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
set_report_vacuum_hook (set_report_vacuum_hook_type) -- called
once per vacuumed relation/index with a PgStat_VacuumRelationCounts
payload tagged by ExtVacReportType (PGSTAT_EXTVAC_TABLE / _INDEX /
_DB / _INVALID).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 234 ++++++++++++++++++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 12 +
src/backend/utils/activity/pgstat_relation.c | 24 ++
src/include/commands/vacuum.h | 29 +++
src/include/pgstat.h | 69 ++++++
6 files changed, 367 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 39395aed0d5..e4d4c93d641 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -283,6 +283,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -410,6 +412,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -485,6 +496,166 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -643,7 +814,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_vacuum_min_duration >= 0));
@@ -660,6 +834,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -687,7 +864,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -803,6 +982,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -985,11 +1167,26 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+
pgstat_progress_end_command();
if (instrument)
@@ -2903,6 +3100,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3015,7 +3213,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3033,6 +3235,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3041,6 +3244,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3065,7 +3276,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3084,12 +3299,21 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 99d0db82ed7..a7fb73173f5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2561,6 +2564,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 41cefcfde54..200f12a2d1b 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1076,6 +1076,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -1084,6 +1086,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -1112,6 +1116,13 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1276,6 +1287,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 92e1f60a080..226d7aa06d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -272,6 +272,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 956d9cea36d..a925f7da992 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -21,9 +21,11 @@
#include "catalog/pg_class.h"
#include "catalog/pg_statistic.h"
#include "catalog/pg_type.h"
+#include "executor/instrument.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -354,6 +356,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7db36cf8add..8d934973dc1 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,64 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -703,6 +761,17 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
From cf8285d7557582d6995d58ca62599e7e47b20b1b Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 28 Apr 2026 03:43:29 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics: extension for extended vacuum
statistics
Introduce a new extension that collects extended per-vacuum
metrics via set_report_vacuum_hook and stores them through pgstat's
custom statistics infrastructure.
Tracking scope is controlled by GUCs:
* vacuum_statistics.enabled -- master switch
* vacuum_statistics.object_types -- databases / relations / all
* vacuum_statistics.track_relations -- system / user / all
* vacuum_statistics.track_{databases,relations}_from_list
-- restrict tracking to objects registered via
add_track_database() / add_track_relation();
removal via remove_track_*() and OAT_DROP hook
* vacuum_statistics.collect -- buffers / wal /
general / timing / all, consulted by ACCUM_IF() to skip
unwanted categories at run time
add_track_* / remove_track_* require superuser or pg_read_all_stats.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 ++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 272 ++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 ++++
.../t/054_vacuum_extending_gucs_test.pl | 279 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1387 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 ++++++
doc/src/sgml/filelist.sgml | 1 +
18 files changed, 3909 insertions(+)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7d91fe77db3..3140f2bf844 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..aa3a9ec9699
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,272 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Track-list mutation requires superuser or pg_read_all_stats; hide the
+-- functions from PUBLIC so the error is also produced for ordinary users
+-- before the C-level privilege check runs.
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) TO pg_read_all_stats;
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT STABLE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..4f8f025c63e
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_frozen_page_marks_cleared(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_visible_page_marks_cleared(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_frozen_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_visible_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..a195249842b
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,279 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra),
+ # Make pending stats visible to subsequent sessions without sleeping.
+ "SELECT pg_stat_force_next_flush();");
+ $node->safe_psql($db, $sql);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session. Assert not only that the
+ # row count is zero, but that the specific counters remain zero: a stray
+ # row with zero counters would otherwise pass a bare COUNT(*)=0 check.
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no rows when disabled');
+
+ my $sums = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_dirtied), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($sums, '0', 'no counters accumulated when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 6: vacuum_statistics.collect - per-category gating
+#
+# With collect='wal' only wal_* counters must advance; buffer, timing, and
+# general categories must stay at zero. With collect='buffers' the inverse
+# holds. Unknown tokens must be rejected by the check-hook.
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.collect' => sub {
+ # wal-only: WAL counters should accumulate, buffers/timing/general should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'wal'"],
+ modify => 1,
+ });
+
+ my $wal = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal, 't', "collect='wal': wal_records accumulated");
+
+ my $other = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0)
+ + COALESCE(SUM(total_time), 0)
+ + COALESCE(SUM(tuples_deleted), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($other, '0',
+ "collect='wal': buffer/timing/general counters not accumulated");
+
+ # buffers-only: buffer counters should advance, WAL should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'buffers'"],
+ modify => 1,
+ });
+
+ my $buf = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($buf, 't', "collect='buffers': buffer counters accumulated");
+
+ my $wal_off = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal_off, '0',
+ "collect='buffers': WAL counters not accumulated");
+
+ # Unknown category must be rejected by the check-hook.
+ my ($ret, $stdout, $stderr) = $node->psql($dbname,
+ "SET vacuum_statistics.collect = 'nope'");
+ isnt($ret, 0, "collect='nope': rejected by check-hook");
+ like($stderr, qr/Unrecognized category "nope"/,
+ "collect='nope': errdetail names the offending token");
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..75d1bd2cf06
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1387 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_authid.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+#include "utils/tuplestore.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/*
+ * Bit flags for evs_collect_mask. Each category groups counters that can be
+ * accumulated (or skipped) together, letting users reduce overhead at run
+ * time by turning off categories they don't need.
+ */
+#define EVS_COLLECT_BUFFERS 0x1 /* blks_*, blk_*_time */
+#define EVS_COLLECT_WAL 0x2 /* wal_records, wal_fpi, wal_bytes */
+#define EVS_COLLECT_GENERAL 0x4 /* tuples_deleted, pages_*, vm_*,
+ * wraparound_failsafe_count,
+ * interrupts_count */
+#define EVS_COLLECT_TIMING 0x8 /* delay_time, total_time */
+#define EVS_COLLECT_ALL (EVS_COLLECT_BUFFERS | EVS_COLLECT_WAL | \
+ EVS_COLLECT_GENERAL | EVS_COLLECT_TIMING)
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+static char *evs_collect = "all"; /* categories to collect */
+static int evs_collect_mask = EVS_COLLECT_ALL;
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+static shmem_request_hook_type prev_shmem_request_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+static void evs_shmem_request(void);
+
+/* Hash tables for track_databases and track_relations_list (backend-local) */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+/*
+ * Named LWLock tranche protecting the on-disk track file and serializing
+ * backend-local reloads/saves across concurrent backends.
+ */
+#define EVS_TRACK_TRANCHE_NAME "ext_vacuum_statistics_track"
+static LWLock *evs_track_lock = NULL;
+
+static inline LWLock *
+evs_get_track_lock(void)
+{
+ if (evs_track_lock == NULL)
+ evs_track_lock = &GetNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME)->lock;
+ return evs_track_lock;
+}
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/*
+ * Accumulate a single counter only if its category is enabled in
+ * evs_collect_mask. Parentheses around every argument: the macro is invoked
+ * from expression contexts and with expressions as the destination pointer.
+ */
+#define ACCUM_IF(dst, src, field, cat) \
+ do { \
+ if ((evs_collect_mask) & (cat)) \
+ ((dst))->field += ((src))->field; \
+ } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_dirtied, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_written, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_fetched, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_read_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_write_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, delay_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, total_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, wal_records, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_fpi, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_bytes, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wraparound_failsafe_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, interrupts_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, tuples_deleted, EVS_COLLECT_GENERAL);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/*
+ * GUC check hooks: validate the string and compute the bitmask into *extra.
+ * Rejecting unknown values here prevents silent fall-through to "all".
+ */
+static bool
+evs_track_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "databases") == 0)
+ *bits = EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "relations") == 0)
+ *bits = EVS_TRACK_RELATIONS;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"databases\", \"relations\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ evs_track_bits = *((int *) extra);
+}
+
+static bool
+evs_track_relations_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+ else if (strcmp(*newval, "system") == 0)
+ *bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(*newval, "user") == 0)
+ *bits = EVS_FILTER_USER;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"system\", \"user\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ evs_track_relations_bits = *((int *) extra);
+}
+
+/*
+ * Check hook for vacuum_statistics.collect.
+ *
+ * Accepts a comma- or whitespace-separated list of category names
+ * (buffers, wal, general, timing) or the shorthand "all". Computes the
+ * matching bitmask once and stashes it in *extra; the assign hook just
+ * copies it into evs_collect_mask. Unknown tokens are rejected so the
+ * setting cannot silently collapse to the "all" default.
+ */
+static bool
+evs_collect_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *mask;
+ char *copy;
+ char *p;
+ char *tok;
+ int accum = 0;
+ bool saw_all = false;
+
+ if (*newval == NULL)
+ return false;
+
+ mask = (int *) guc_malloc(LOG, sizeof(int));
+ if (!mask)
+ return false;
+
+ /* Empty string means "all", matching the default behavior. */
+ if ((*newval)[0] == '\0')
+ {
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+ }
+
+ copy = pstrdup(*newval);
+ for (p = copy; (tok = strtok(p, " \t,")) != NULL; p = NULL)
+ {
+ if (pg_strcasecmp(tok, "all") == 0)
+ saw_all = true;
+ else if (pg_strcasecmp(tok, "buffers") == 0)
+ accum |= EVS_COLLECT_BUFFERS;
+ else if (pg_strcasecmp(tok, "wal") == 0)
+ accum |= EVS_COLLECT_WAL;
+ else if (pg_strcasecmp(tok, "general") == 0)
+ accum |= EVS_COLLECT_GENERAL;
+ else if (pg_strcasecmp(tok, "timing") == 0)
+ accum |= EVS_COLLECT_TIMING;
+ else
+ {
+ /*
+ * GUC_check_errdetail formats the message immediately, but tok
+ * points into copy; emit the detail first, then free the
+ * scratch buffer so the formatted string is already stashed in
+ * GUC_check_errdetail_string.
+ */
+ GUC_check_errdetail("Unrecognized category \"%s\" in vacuum_statistics.collect; "
+ "allowed values are \"all\", \"buffers\", \"wal\", \"general\", \"timing\".",
+ tok);
+ pfree(copy);
+ guc_free(mask);
+ return false;
+ }
+ }
+ pfree(copy);
+
+ *mask = saw_all ? EVS_COLLECT_ALL : accum;
+ if (*mask == 0)
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+}
+
+static void
+evs_collect_assign_hook(const char *newval, void *extra)
+{
+ evs_collect_mask = *((int *) extra);
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0,
+ evs_track_check_hook,
+ evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0,
+ evs_track_relations_check_hook,
+ evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.collect",
+ "Statistics categories to collect.",
+ "Comma- or whitespace-separated list of: "
+ "\"buffers\", \"wal\", \"general\", \"timing\"; "
+ "or \"all\" for every category (default).",
+ &evs_collect, "all",
+ PGC_SUSET, 0,
+ evs_collect_check_hook,
+ evs_collect_assign_hook, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = evs_shmem_request;
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+static void
+evs_shmem_request(void)
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
+ RequestNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME, 1);
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ LWLock *lock = evs_get_track_lock();
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ LWLock *lock = evs_get_track_lock();
+ bool found;
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+/*
+ * Initialize the backend-local tracking hashes and load their contents
+ * from the on-disk file.
+ *
+ * The hashes are per-backend, so no lock is needed to protect them from
+ * other processes; however, another backend may be concurrently rewriting
+ * the track file, so we take a shared lock for the file read.
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+ LWLock *lock;
+ bool need_load;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ lock = evs_get_track_lock();
+
+ if (evs_track_databases_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_databases_hash =
+ hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ if (evs_track_relations_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_relations_hash =
+ hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ need_load = !LWLockHeldByMe(lock);
+ if (need_load)
+ LWLockAcquire(lock, LW_SHARED);
+ PG_TRY();
+ {
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+ }
+ PG_FINALLY();
+ {
+ if (need_load)
+ LWLockRelease(lock);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Load track lists from disk into the backend-local hashes.
+ *
+ * Caller must hold evs_track_lock at least in shared mode, since the file
+ * may be concurrently rewritten by another backend.
+ */
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[MAXPGPATH];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open track file \"%s\": %m", path)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ size_t len = strlen(buf);
+
+ /* Reject unterminated lines (longer than buffer) as corruption. */
+ if (len > 0 && buf[len - 1] != '\n' && !feof(fp))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("line too long in track file \"%s\"", path)));
+
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+
+ if (ferror(fp))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read track file \"%s\": %m", path)));
+ }
+ PG_FINALLY();
+ {
+ FreeFile(fp);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Atomically rewrite the track file. Caller must hold evs_track_lock
+ * in exclusive mode.
+ */
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+ bool failed = false;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s.tmp", path);
+
+ fp = AllocateFile(tmppath, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create track file \"%s\": %m", tmppath)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ if (fputs("[databases]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ if (fprintf(fp, "%u\n", *entry) < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fputs("[relations]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ int rc;
+
+ if (OidIsValid(rel_entry->dboid))
+ rc = fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ rc = fprintf(fp, "0 %u\n", rel_entry->reloid);
+ if (rc < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fflush(fp) != 0)
+ failed = true;
+
+ if (!failed)
+ {
+ int fd = fileno(fp);
+
+ if (fd >= 0 && pg_fsync(fd) != 0)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not fsync track file \"%s\": %m",
+ tmppath)));
+ }
+ }
+ PG_CATCH();
+ {
+ FreeFile(fp);
+ (void) unlink(tmppath);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (FreeFile(fp) != 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close track file \"%s\": %m", tmppath)));
+ failed = true;
+ }
+
+ if (failed)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write track file \"%s\": %m", tmppath)));
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ return;
+ }
+
+ if (durable_rename(tmppath, path, LOG) != 0)
+ {
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ }
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+/*
+ * Mutating track-list entry points: require server-wide privilege, since
+ * the underlying lists steer tracking for every backend.
+ */
+static void
+evs_require_track_privilege(const char *funcname)
+{
+ if (!superuser() && !has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS))
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for function %s", funcname),
+ errhint("Only superusers and members of pg_read_all_stats "
+ "may change the vacuum statistics track list.")));
+}
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid,
+ InvalidOid, NULL);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index ebb7f83d8c5..d7dc0fd07f0 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index b9b03654aad..2a38f9042bb 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 25a85082759..85d721467c0 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v38-0001-Track-table-VM-stability.patch (21.7K, 3-v38-0001-Track-table-VM-stability.patch)
download | inline diff:
From 19f5a39f7e97d3fc2d18415ba2c51ffcd3b32f49 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH 1/3] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
[text/plain] v38-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch (25.0K, 4-v38-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch)
download | inline diff:
From 3a5e0bd82578d1fea63d6bda229dc4d0b224684e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add infrastructure inside lazy vacuum to gather extended per-vacuum
metrics and expose them to extensions via a new hook. Core itself
does not persist these metrics — that is the job of an extension
(see ext_vacuum_statistics).
Statistics are gathered separately for tables and indexes according
to vacuum phases. The ExtVacReport union and type field distinguish
PGSTAT_EXTVAC_TABLE vs PGSTAT_EXTVAC_INDEX. Heap vacuum stats are
sent to the cumulative statistics system after vacuum has processed
the indexes. Database vacuum statistics aggregate per-table and
per-index statistics within the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
set_report_vacuum_hook (set_report_vacuum_hook_type) -- called
once per vacuumed relation/index with a PgStat_VacuumRelationCounts
payload tagged by ExtVacReportType (PGSTAT_EXTVAC_TABLE / _INDEX /
_DB / _INVALID).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 234 ++++++++++++++++++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 12 +
src/backend/utils/activity/pgstat_relation.c | 24 ++
src/include/commands/vacuum.h | 29 +++
src/include/pgstat.h | 69 ++++++
6 files changed, 367 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 39395aed0d5..e4d4c93d641 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -283,6 +283,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -410,6 +412,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -485,6 +496,166 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -643,7 +814,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_vacuum_min_duration >= 0));
@@ -660,6 +834,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -687,7 +864,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -803,6 +982,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -985,11 +1167,26 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+
pgstat_progress_end_command();
if (instrument)
@@ -2903,6 +3100,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3015,7 +3213,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3033,6 +3235,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3041,6 +3244,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3065,7 +3276,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3084,12 +3299,21 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 99d0db82ed7..a7fb73173f5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2561,6 +2564,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 41cefcfde54..200f12a2d1b 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1076,6 +1076,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -1084,6 +1086,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -1112,6 +1116,13 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1276,6 +1287,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 92e1f60a080..226d7aa06d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -272,6 +272,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 956d9cea36d..a925f7da992 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -21,9 +21,11 @@
#include "catalog/pg_class.h"
#include "catalog/pg_statistic.h"
#include "catalog/pg_type.h"
+#include "executor/instrument.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -354,6 +356,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7db36cf8add..8d934973dc1 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,64 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -703,6 +761,17 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
[text/plain] v38-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch (145.2K, 5-v38-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch)
download | inline diff:
From cf8285d7557582d6995d58ca62599e7e47b20b1b Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 28 Apr 2026 03:43:29 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics: extension for extended vacuum
statistics
Introduce a new extension that collects extended per-vacuum
metrics via set_report_vacuum_hook and stores them through pgstat's
custom statistics infrastructure.
Tracking scope is controlled by GUCs:
* vacuum_statistics.enabled -- master switch
* vacuum_statistics.object_types -- databases / relations / all
* vacuum_statistics.track_relations -- system / user / all
* vacuum_statistics.track_{databases,relations}_from_list
-- restrict tracking to objects registered via
add_track_database() / add_track_relation();
removal via remove_track_*() and OAT_DROP hook
* vacuum_statistics.collect -- buffers / wal /
general / timing / all, consulted by ACCUM_IF() to skip
unwanted categories at run time
add_track_* / remove_track_* require superuser or pg_read_all_stats.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 ++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 272 ++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 ++++
.../t/054_vacuum_extending_gucs_test.pl | 279 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1387 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 ++++++
doc/src/sgml/filelist.sgml | 1 +
18 files changed, 3909 insertions(+)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7d91fe77db3..3140f2bf844 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..aa3a9ec9699
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,272 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Track-list mutation requires superuser or pg_read_all_stats; hide the
+-- functions from PUBLIC so the error is also produced for ordinary users
+-- before the C-level privilege check runs.
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) TO pg_read_all_stats;
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT STABLE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..4f8f025c63e
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_frozen_page_marks_cleared(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_visible_page_marks_cleared(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_frozen_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_visible_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..a195249842b
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,279 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra),
+ # Make pending stats visible to subsequent sessions without sleeping.
+ "SELECT pg_stat_force_next_flush();");
+ $node->safe_psql($db, $sql);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session. Assert not only that the
+ # row count is zero, but that the specific counters remain zero: a stray
+ # row with zero counters would otherwise pass a bare COUNT(*)=0 check.
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no rows when disabled');
+
+ my $sums = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_dirtied), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($sums, '0', 'no counters accumulated when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 6: vacuum_statistics.collect - per-category gating
+#
+# With collect='wal' only wal_* counters must advance; buffer, timing, and
+# general categories must stay at zero. With collect='buffers' the inverse
+# holds. Unknown tokens must be rejected by the check-hook.
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.collect' => sub {
+ # wal-only: WAL counters should accumulate, buffers/timing/general should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'wal'"],
+ modify => 1,
+ });
+
+ my $wal = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal, 't', "collect='wal': wal_records accumulated");
+
+ my $other = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0)
+ + COALESCE(SUM(total_time), 0)
+ + COALESCE(SUM(tuples_deleted), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($other, '0',
+ "collect='wal': buffer/timing/general counters not accumulated");
+
+ # buffers-only: buffer counters should advance, WAL should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'buffers'"],
+ modify => 1,
+ });
+
+ my $buf = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($buf, 't', "collect='buffers': buffer counters accumulated");
+
+ my $wal_off = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal_off, '0',
+ "collect='buffers': WAL counters not accumulated");
+
+ # Unknown category must be rejected by the check-hook.
+ my ($ret, $stdout, $stderr) = $node->psql($dbname,
+ "SET vacuum_statistics.collect = 'nope'");
+ isnt($ret, 0, "collect='nope': rejected by check-hook");
+ like($stderr, qr/Unrecognized category "nope"/,
+ "collect='nope': errdetail names the offending token");
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..75d1bd2cf06
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1387 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_authid.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+#include "utils/tuplestore.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/*
+ * Bit flags for evs_collect_mask. Each category groups counters that can be
+ * accumulated (or skipped) together, letting users reduce overhead at run
+ * time by turning off categories they don't need.
+ */
+#define EVS_COLLECT_BUFFERS 0x1 /* blks_*, blk_*_time */
+#define EVS_COLLECT_WAL 0x2 /* wal_records, wal_fpi, wal_bytes */
+#define EVS_COLLECT_GENERAL 0x4 /* tuples_deleted, pages_*, vm_*,
+ * wraparound_failsafe_count,
+ * interrupts_count */
+#define EVS_COLLECT_TIMING 0x8 /* delay_time, total_time */
+#define EVS_COLLECT_ALL (EVS_COLLECT_BUFFERS | EVS_COLLECT_WAL | \
+ EVS_COLLECT_GENERAL | EVS_COLLECT_TIMING)
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+static char *evs_collect = "all"; /* categories to collect */
+static int evs_collect_mask = EVS_COLLECT_ALL;
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+static shmem_request_hook_type prev_shmem_request_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+static void evs_shmem_request(void);
+
+/* Hash tables for track_databases and track_relations_list (backend-local) */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+/*
+ * Named LWLock tranche protecting the on-disk track file and serializing
+ * backend-local reloads/saves across concurrent backends.
+ */
+#define EVS_TRACK_TRANCHE_NAME "ext_vacuum_statistics_track"
+static LWLock *evs_track_lock = NULL;
+
+static inline LWLock *
+evs_get_track_lock(void)
+{
+ if (evs_track_lock == NULL)
+ evs_track_lock = &GetNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME)->lock;
+ return evs_track_lock;
+}
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/*
+ * Accumulate a single counter only if its category is enabled in
+ * evs_collect_mask. Parentheses around every argument: the macro is invoked
+ * from expression contexts and with expressions as the destination pointer.
+ */
+#define ACCUM_IF(dst, src, field, cat) \
+ do { \
+ if ((evs_collect_mask) & (cat)) \
+ ((dst))->field += ((src))->field; \
+ } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_dirtied, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_written, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_fetched, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_read_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_write_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, delay_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, total_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, wal_records, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_fpi, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_bytes, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wraparound_failsafe_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, interrupts_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, tuples_deleted, EVS_COLLECT_GENERAL);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/*
+ * GUC check hooks: validate the string and compute the bitmask into *extra.
+ * Rejecting unknown values here prevents silent fall-through to "all".
+ */
+static bool
+evs_track_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "databases") == 0)
+ *bits = EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "relations") == 0)
+ *bits = EVS_TRACK_RELATIONS;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"databases\", \"relations\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ evs_track_bits = *((int *) extra);
+}
+
+static bool
+evs_track_relations_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+ else if (strcmp(*newval, "system") == 0)
+ *bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(*newval, "user") == 0)
+ *bits = EVS_FILTER_USER;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"system\", \"user\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ evs_track_relations_bits = *((int *) extra);
+}
+
+/*
+ * Check hook for vacuum_statistics.collect.
+ *
+ * Accepts a comma- or whitespace-separated list of category names
+ * (buffers, wal, general, timing) or the shorthand "all". Computes the
+ * matching bitmask once and stashes it in *extra; the assign hook just
+ * copies it into evs_collect_mask. Unknown tokens are rejected so the
+ * setting cannot silently collapse to the "all" default.
+ */
+static bool
+evs_collect_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *mask;
+ char *copy;
+ char *p;
+ char *tok;
+ int accum = 0;
+ bool saw_all = false;
+
+ if (*newval == NULL)
+ return false;
+
+ mask = (int *) guc_malloc(LOG, sizeof(int));
+ if (!mask)
+ return false;
+
+ /* Empty string means "all", matching the default behavior. */
+ if ((*newval)[0] == '\0')
+ {
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+ }
+
+ copy = pstrdup(*newval);
+ for (p = copy; (tok = strtok(p, " \t,")) != NULL; p = NULL)
+ {
+ if (pg_strcasecmp(tok, "all") == 0)
+ saw_all = true;
+ else if (pg_strcasecmp(tok, "buffers") == 0)
+ accum |= EVS_COLLECT_BUFFERS;
+ else if (pg_strcasecmp(tok, "wal") == 0)
+ accum |= EVS_COLLECT_WAL;
+ else if (pg_strcasecmp(tok, "general") == 0)
+ accum |= EVS_COLLECT_GENERAL;
+ else if (pg_strcasecmp(tok, "timing") == 0)
+ accum |= EVS_COLLECT_TIMING;
+ else
+ {
+ /*
+ * GUC_check_errdetail formats the message immediately, but tok
+ * points into copy; emit the detail first, then free the
+ * scratch buffer so the formatted string is already stashed in
+ * GUC_check_errdetail_string.
+ */
+ GUC_check_errdetail("Unrecognized category \"%s\" in vacuum_statistics.collect; "
+ "allowed values are \"all\", \"buffers\", \"wal\", \"general\", \"timing\".",
+ tok);
+ pfree(copy);
+ guc_free(mask);
+ return false;
+ }
+ }
+ pfree(copy);
+
+ *mask = saw_all ? EVS_COLLECT_ALL : accum;
+ if (*mask == 0)
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+}
+
+static void
+evs_collect_assign_hook(const char *newval, void *extra)
+{
+ evs_collect_mask = *((int *) extra);
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0,
+ evs_track_check_hook,
+ evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0,
+ evs_track_relations_check_hook,
+ evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.collect",
+ "Statistics categories to collect.",
+ "Comma- or whitespace-separated list of: "
+ "\"buffers\", \"wal\", \"general\", \"timing\"; "
+ "or \"all\" for every category (default).",
+ &evs_collect, "all",
+ PGC_SUSET, 0,
+ evs_collect_check_hook,
+ evs_collect_assign_hook, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = evs_shmem_request;
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+static void
+evs_shmem_request(void)
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
+ RequestNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME, 1);
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ LWLock *lock = evs_get_track_lock();
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ LWLock *lock = evs_get_track_lock();
+ bool found;
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+/*
+ * Initialize the backend-local tracking hashes and load their contents
+ * from the on-disk file.
+ *
+ * The hashes are per-backend, so no lock is needed to protect them from
+ * other processes; however, another backend may be concurrently rewriting
+ * the track file, so we take a shared lock for the file read.
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+ LWLock *lock;
+ bool need_load;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ lock = evs_get_track_lock();
+
+ if (evs_track_databases_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_databases_hash =
+ hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ if (evs_track_relations_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_relations_hash =
+ hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ need_load = !LWLockHeldByMe(lock);
+ if (need_load)
+ LWLockAcquire(lock, LW_SHARED);
+ PG_TRY();
+ {
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+ }
+ PG_FINALLY();
+ {
+ if (need_load)
+ LWLockRelease(lock);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Load track lists from disk into the backend-local hashes.
+ *
+ * Caller must hold evs_track_lock at least in shared mode, since the file
+ * may be concurrently rewritten by another backend.
+ */
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[MAXPGPATH];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open track file \"%s\": %m", path)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ size_t len = strlen(buf);
+
+ /* Reject unterminated lines (longer than buffer) as corruption. */
+ if (len > 0 && buf[len - 1] != '\n' && !feof(fp))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("line too long in track file \"%s\"", path)));
+
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+
+ if (ferror(fp))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read track file \"%s\": %m", path)));
+ }
+ PG_FINALLY();
+ {
+ FreeFile(fp);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Atomically rewrite the track file. Caller must hold evs_track_lock
+ * in exclusive mode.
+ */
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+ bool failed = false;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s.tmp", path);
+
+ fp = AllocateFile(tmppath, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create track file \"%s\": %m", tmppath)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ if (fputs("[databases]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ if (fprintf(fp, "%u\n", *entry) < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fputs("[relations]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ int rc;
+
+ if (OidIsValid(rel_entry->dboid))
+ rc = fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ rc = fprintf(fp, "0 %u\n", rel_entry->reloid);
+ if (rc < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fflush(fp) != 0)
+ failed = true;
+
+ if (!failed)
+ {
+ int fd = fileno(fp);
+
+ if (fd >= 0 && pg_fsync(fd) != 0)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not fsync track file \"%s\": %m",
+ tmppath)));
+ }
+ }
+ PG_CATCH();
+ {
+ FreeFile(fp);
+ (void) unlink(tmppath);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (FreeFile(fp) != 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close track file \"%s\": %m", tmppath)));
+ failed = true;
+ }
+
+ if (failed)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write track file \"%s\": %m", tmppath)));
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ return;
+ }
+
+ if (durable_rename(tmppath, path, LOG) != 0)
+ {
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ }
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+/*
+ * Mutating track-list entry points: require server-wide privilege, since
+ * the underlying lists steer tracking for every backend.
+ */
+static void
+evs_require_track_privilege(const char *funcname)
+{
+ if (!superuser() && !has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS))
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for function %s", funcname),
+ errhint("Only superusers and members of pg_read_all_stats "
+ "may change the vacuum statistics track list.")));
+}
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid,
+ InvalidOid, NULL);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index ebb7f83d8c5..d7dc0fd07f0 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index b9b03654aad..2a38f9042bb 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 25a85082759..85d721467c0 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 77+ messages in thread
* Re: Vacuum statistics
@ 2026-04-28 05:28 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 0 replies; 77+ messages in thread
From: Alena Rybakina @ 2026-04-28 05:28 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrey Borodin <[email protected]>; Andrei Zubkov <[email protected]>; Andrei Lepikhov <[email protected]>
On 28.04.2026 05:16, Alena Rybakina wrote:
> Hi, all!
>
> I have updated the core patch that implements the machinery for
> collecting extended vacuum statistics (I didn't touch the first patch
> that is ready for commit, only patches that are related to extension),
> and rebased the ext_vacuum_statistics extension on top of it. The
> split is intentional: the core only gathers metrics and hands them
> out, while the actual storage and SQL-level access to the statistics
> live entirely in the extension. If the extension is not loaded, the
> overhead is essentially zero - we only fill a small struct on the
> stack and do a NULL check on the hook.
>
> What was updated in the core
>
> The core gains the machinery and the hook through which the extension
> receives metrics after each vacuum.
>
> The hook. A new hook has been added in pgstat -
> set_report_vacuum_hook. It is fired once per vacuumed table and once
> per vacuumed index, plus when forming the per-database aggregate. The
> extension registers its handler in _PG_init and by default the hook is
> NULL, so without an extension the core behaves exactly as before.
>
> The set of statistics is the same as before. Common to tables, indexes
> and the database - hits and misses in shared buffers, number of
> dirtied and written pages, WAL volume, buffer read and write times,
> sleep time spent in delay points, total wall-clock vacuum time
> (including I/O and lock waits), counter of emergency anti-wraparound
> vacuums, number of interrupts and removed tuples. Tables additionally
> report frozen tuples, pages marked all-frozen / all-visible in the
> visibility map, number of scanned and removed pages, number of index
> passes, etc. Indexes report freed pages.
>
> The least obvious part of the implementation is subtracting index
> statistics from the table statistics. This is the bit worth
> highlighting. The thing is that indexes are vacuumed before the heap,
> and the buffer and WAL statistics that we capture at the heap level by
> the end of the heap vacuum already include everything that was spent
> on the indexes. If we simply expose the diff of
> pgBufferUsage/pgWalUsage between start and end, the table ends up with
> double-counted pages/WAL: once in its own report, and a second time
> inside the reports of its indexes. This is especially noticeable with
> parallel index vacuum: workers accumulate their usage in the leader
> only after they finish, so without subtraction the heap report would
> receive the combined cost of all workers as a "bonus".
>
> To handle this, as each index finishes vacuuming, its counters are
> accumulated into the state of the current operation, and at the moment
> the heap report is built these sums are subtracted out. As a result,
> the extension receives clean numbers: "this is what was actually spent
> on the table itself", and separately "this is what was actually spent
> on each index". The behaviour is idempotent for both serial and
> parallel vacuum.
>
> The ext_vacuum_statistics extension
>
> The extension registers the hook handler and stores the received data
> through the pgstat custom statistics infrastructure. That is, vacuum
> counters are kept not in the extension's own files, but together with
> the regular cumulative statistics - they survive a restart and are
> reset together with pg_stat_reset_*. Access is provided through three
> views: one for tables, one for indexes, and one with the per-database
> aggregate.
>
> Filtering
>
> This is where the main flexibility lives - the extension does not
> force "collect everything", but lets you choose both what to track and
> which metrics to keep.
>
> By object type. You can limit collection to databases only (without
> per-table detail), to tables only, or collect both. Among tables, you
> can additionally filter system / user / all.
>
> By an explicit list. An alternative to "by type" is a whitelist: you
> turn the corresponding mode on, and the extension starts collecting
> statistics only for the databases and tables that were explicitly
> registered via add_track_database / add_track_relation (with matching
> remove_* for removal). When the lists are off, the type filter is in
> effect; when they are on, only the list applies. This is convenient
> when you are interested in monitoring specific "hot" tables and do not
> want to spend memory on statistics for everything else.
> This list is persisted to disk, and there is one more non-trivial part
> here. List changes are concurrent - multiple sessions may call
> add_track_* simultaneously, plus there is an object-access hook that
> cleans the entry on DROP. To avoid ending up with a torn file, access
> to the list is serialized via a dedicated LWLock tranche (requested
> from a shmem_request_hook), and the file itself is written atomically:
> first into a temporary file, then fflush + pg_fsync + durable_rename.
> All I/O return codes are checked; on error the temporary file is
> removed and the real one is left untouched; PG_TRY/PG_CATCH guarantees
> cleanup on ereport(ERROR). Reading the list takes the same lock in
> shared mode, so a concurrent write cannot tear the load.
>
> By metric category. There is also a GUC that takes a list and turns on
> the categories of interest - buffers, WAL, general counters, timings
> (or all). Unwanted categories are simply skipped on the hook handler
> side and never make it into the pgstat entry, which reduces the
> overhead of the handler itself. This is useful when, for example, only
> timings are needed - in that case the extension does not waste time
> copying the buffer and WAL fields.
>
> Privileges. The add_track_* / remove_track_* functions require
> superuser or pg_read_all_stats. At the SQL level, EXECUTE is revoked
> from PUBLIC and granted only to pg_read_all_stats, so a regular user
> has no access to mutating the list. The views are unrestricted, like
> regular statistics.
>
> What is in the patches
>
> 0002-Machinery-for-grabbing-extended-vacuum-statistics.patch - the
> machinery in the core plus the hook.
> 0003-ext_vacuum_statistics-...patch - the extension itself, filtering,
> views, tests.
>
I noticed CI's complaints during extension installation and fixed it.
--
-----------
Best regards,
Alena Rybakina
Yandex Cloud
From 19f5a39f7e97d3fc2d18415ba2c51ffcd3b32f49 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH 1/3] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
From 3a5e0bd82578d1fea63d6bda229dc4d0b224684e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add infrastructure inside lazy vacuum to gather extended per-vacuum
metrics and expose them to extensions via a new hook. Core itself
does not persist these metrics — that is the job of an extension
(see ext_vacuum_statistics).
Statistics are gathered separately for tables and indexes according
to vacuum phases. The ExtVacReport union and type field distinguish
PGSTAT_EXTVAC_TABLE vs PGSTAT_EXTVAC_INDEX. Heap vacuum stats are
sent to the cumulative statistics system after vacuum has processed
the indexes. Database vacuum statistics aggregate per-table and
per-index statistics within the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
set_report_vacuum_hook (set_report_vacuum_hook_type) -- called
once per vacuumed relation/index with a PgStat_VacuumRelationCounts
payload tagged by ExtVacReportType (PGSTAT_EXTVAC_TABLE / _INDEX /
_DB / _INVALID).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 234 ++++++++++++++++++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 12 +
src/backend/utils/activity/pgstat_relation.c | 24 ++
src/include/commands/vacuum.h | 29 +++
src/include/pgstat.h | 69 ++++++
6 files changed, 367 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 39395aed0d5..e4d4c93d641 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -283,6 +283,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -410,6 +412,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -485,6 +496,166 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -643,7 +814,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_vacuum_min_duration >= 0));
@@ -660,6 +834,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -687,7 +864,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -803,6 +982,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -985,11 +1167,26 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+
pgstat_progress_end_command();
if (instrument)
@@ -2903,6 +3100,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3015,7 +3213,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3033,6 +3235,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3041,6 +3244,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3065,7 +3276,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3084,12 +3299,21 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 99d0db82ed7..a7fb73173f5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2561,6 +2564,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 41cefcfde54..200f12a2d1b 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1076,6 +1076,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -1084,6 +1086,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -1112,6 +1116,13 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1276,6 +1287,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 92e1f60a080..226d7aa06d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -272,6 +272,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 956d9cea36d..a925f7da992 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -21,9 +21,11 @@
#include "catalog/pg_class.h"
#include "catalog/pg_statistic.h"
#include "catalog/pg_type.h"
+#include "executor/instrument.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -354,6 +356,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7db36cf8add..8d934973dc1 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,64 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -703,6 +761,17 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
From 3011a3cfd9ee3d6e4d1c5a12e3d9984f6b6a194e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 28 Apr 2026 03:43:29 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics: extension for extended vacuum
statistics
Introduce a new extension that collects extended per-vacuum
metrics via set_report_vacuum_hook and stores them through pgstat's
custom statistics infrastructure.
Tracking scope is controlled by GUCs:
* vacuum_statistics.enabled -- master switch
* vacuum_statistics.object_types -- databases / relations / all
* vacuum_statistics.track_relations -- system / user / all
* vacuum_statistics.track_{databases,relations}_from_list
-- restrict tracking to objects registered via
add_track_database() / add_track_relation();
removal via remove_track_*() and OAT_DROP hook
* vacuum_statistics.collect -- buffers / wal /
general / timing / all, consulted by ACCUM_IF() to skip
unwanted categories at run time
add_track_* / remove_track_* require superuser or pg_read_all_stats.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 ++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 272 ++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 ++++
.../t/054_vacuum_extending_gucs_test.pl | 279 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1387 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 ++++++
doc/src/sgml/filelist.sgml | 1 +
18 files changed, 3909 insertions(+)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7d91fe77db3..3140f2bf844 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..aa3a9ec9699
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,272 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Track-list mutation requires superuser or pg_read_all_stats; hide the
+-- functions from PUBLIC so the error is also produced for ordinary users
+-- before the C-level privilege check runs.
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) TO pg_read_all_stats;
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT STABLE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..4f8f025c63e
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_frozen_page_marks_cleared(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_visible_page_marks_cleared(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_frozen_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_visible_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..a195249842b
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,279 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra),
+ # Make pending stats visible to subsequent sessions without sleeping.
+ "SELECT pg_stat_force_next_flush();");
+ $node->safe_psql($db, $sql);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session. Assert not only that the
+ # row count is zero, but that the specific counters remain zero: a stray
+ # row with zero counters would otherwise pass a bare COUNT(*)=0 check.
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no rows when disabled');
+
+ my $sums = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_dirtied), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($sums, '0', 'no counters accumulated when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 6: vacuum_statistics.collect - per-category gating
+#
+# With collect='wal' only wal_* counters must advance; buffer, timing, and
+# general categories must stay at zero. With collect='buffers' the inverse
+# holds. Unknown tokens must be rejected by the check-hook.
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.collect' => sub {
+ # wal-only: WAL counters should accumulate, buffers/timing/general should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'wal'"],
+ modify => 1,
+ });
+
+ my $wal = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal, 't', "collect='wal': wal_records accumulated");
+
+ my $other = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0)
+ + COALESCE(SUM(total_time), 0)
+ + COALESCE(SUM(tuples_deleted), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($other, '0',
+ "collect='wal': buffer/timing/general counters not accumulated");
+
+ # buffers-only: buffer counters should advance, WAL should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'buffers'"],
+ modify => 1,
+ });
+
+ my $buf = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($buf, 't', "collect='buffers': buffer counters accumulated");
+
+ my $wal_off = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal_off, '0',
+ "collect='buffers': WAL counters not accumulated");
+
+ # Unknown category must be rejected by the check-hook.
+ my ($ret, $stdout, $stderr) = $node->psql($dbname,
+ "SET vacuum_statistics.collect = 'nope'");
+ isnt($ret, 0, "collect='nope': rejected by check-hook");
+ like($stderr, qr/Unrecognized category "nope"/,
+ "collect='nope': errdetail names the offending token");
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..144b9bcb814
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1387 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_authid.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+#include "utils/tuplestore.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/*
+ * Bit flags for evs_collect_mask. Each category groups counters that can be
+ * accumulated (or skipped) together, letting users reduce overhead at run
+ * time by turning off categories they don't need.
+ */
+#define EVS_COLLECT_BUFFERS 0x1 /* blks_*, blk_*_time */
+#define EVS_COLLECT_WAL 0x2 /* wal_records, wal_fpi, wal_bytes */
+#define EVS_COLLECT_GENERAL 0x4 /* tuples_deleted, pages_*, vm_*,
+ * wraparound_failsafe_count,
+ * interrupts_count */
+#define EVS_COLLECT_TIMING 0x8 /* delay_time, total_time */
+#define EVS_COLLECT_ALL (EVS_COLLECT_BUFFERS | EVS_COLLECT_WAL | \
+ EVS_COLLECT_GENERAL | EVS_COLLECT_TIMING)
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+static char *evs_collect = "all"; /* categories to collect */
+static int evs_collect_mask = EVS_COLLECT_ALL;
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+static shmem_request_hook_type prev_shmem_request_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+static void evs_shmem_request(void);
+
+/* Hash tables for track_databases and track_relations_list (backend-local) */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+/*
+ * Named LWLock tranche protecting the on-disk track file and serializing
+ * backend-local reloads/saves across concurrent backends.
+ */
+#define EVS_TRACK_TRANCHE_NAME "ext_vacuum_statistics_track"
+static LWLock *evs_track_lock = NULL;
+
+static inline LWLock *
+evs_get_track_lock(void)
+{
+ if (evs_track_lock == NULL)
+ evs_track_lock = &GetNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME)->lock;
+ return evs_track_lock;
+}
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/*
+ * Accumulate a single counter only if its category is enabled in
+ * evs_collect_mask. Parentheses around every argument: the macro is invoked
+ * from expression contexts and with expressions as the destination pointer.
+ */
+#define ACCUM_IF(dst, src, field, cat) \
+ do { \
+ if ((evs_collect_mask) & (cat)) \
+ ((dst))->field += ((src))->field; \
+ } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_dirtied, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_written, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_fetched, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_read_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_write_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, delay_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, total_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, wal_records, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_fpi, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_bytes, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wraparound_failsafe_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, interrupts_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, tuples_deleted, EVS_COLLECT_GENERAL);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/*
+ * GUC check hooks: validate the string and compute the bitmask into *extra.
+ * Rejecting unknown values here prevents silent fall-through to "all".
+ */
+static bool
+evs_track_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "databases") == 0)
+ *bits = EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "relations") == 0)
+ *bits = EVS_TRACK_RELATIONS;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"databases\", \"relations\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ evs_track_bits = *((int *) extra);
+}
+
+static bool
+evs_track_relations_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+ else if (strcmp(*newval, "system") == 0)
+ *bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(*newval, "user") == 0)
+ *bits = EVS_FILTER_USER;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"system\", \"user\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ evs_track_relations_bits = *((int *) extra);
+}
+
+/*
+ * Check hook for vacuum_statistics.collect.
+ *
+ * Accepts a comma- or whitespace-separated list of category names
+ * (buffers, wal, general, timing) or the shorthand "all". Computes the
+ * matching bitmask once and stashes it in *extra; the assign hook just
+ * copies it into evs_collect_mask. Unknown tokens are rejected so the
+ * setting cannot silently collapse to the "all" default.
+ */
+static bool
+evs_collect_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *mask;
+ char *copy;
+ char *p;
+ char *tok;
+ int accum = 0;
+ bool saw_all = false;
+
+ if (*newval == NULL)
+ return false;
+
+ mask = (int *) guc_malloc(LOG, sizeof(int));
+ if (!mask)
+ return false;
+
+ /* Empty string means "all", matching the default behavior. */
+ if ((*newval)[0] == '\0')
+ {
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+ }
+
+ copy = pstrdup(*newval);
+ for (p = copy; (tok = strtok(p, " \t,")) != NULL; p = NULL)
+ {
+ if (pg_strcasecmp(tok, "all") == 0)
+ saw_all = true;
+ else if (pg_strcasecmp(tok, "buffers") == 0)
+ accum |= EVS_COLLECT_BUFFERS;
+ else if (pg_strcasecmp(tok, "wal") == 0)
+ accum |= EVS_COLLECT_WAL;
+ else if (pg_strcasecmp(tok, "general") == 0)
+ accum |= EVS_COLLECT_GENERAL;
+ else if (pg_strcasecmp(tok, "timing") == 0)
+ accum |= EVS_COLLECT_TIMING;
+ else
+ {
+ /*
+ * GUC_check_errdetail formats the message immediately, but tok
+ * points into copy; emit the detail first, then free the
+ * scratch buffer so the formatted string is already stashed in
+ * GUC_check_errdetail_string.
+ */
+ GUC_check_errdetail("Unrecognized category \"%s\" in vacuum_statistics.collect; "
+ "allowed values are \"all\", \"buffers\", \"wal\", \"general\", \"timing\".",
+ tok);
+ pfree(copy);
+ guc_free(mask);
+ return false;
+ }
+ }
+ pfree(copy);
+
+ *mask = saw_all ? EVS_COLLECT_ALL : accum;
+ if (*mask == 0)
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+}
+
+static void
+evs_collect_assign_hook(const char *newval, void *extra)
+{
+ evs_collect_mask = *((int *) extra);
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0,
+ evs_track_check_hook,
+ evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0,
+ evs_track_relations_check_hook,
+ evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.collect",
+ "Statistics categories to collect.",
+ "Comma- or whitespace-separated list of: "
+ "\"buffers\", \"wal\", \"general\", \"timing\"; "
+ "or \"all\" for every category (default).",
+ &evs_collect, "all",
+ PGC_SUSET, 0,
+ evs_collect_check_hook,
+ evs_collect_assign_hook, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = evs_shmem_request;
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+static void
+evs_shmem_request(void)
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
+ RequestNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME, 1);
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ LWLock *lock = evs_get_track_lock();
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ LWLock *lock = evs_get_track_lock();
+ bool found;
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+/*
+ * Initialize the backend-local tracking hashes and load their contents
+ * from the on-disk file.
+ *
+ * The hashes are per-backend, so no lock is needed to protect them from
+ * other processes; however, another backend may be concurrently rewriting
+ * the track file, so we take a shared lock for the file read.
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+ LWLock *lock;
+ bool need_load;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ lock = evs_get_track_lock();
+
+ if (evs_track_databases_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_databases_hash =
+ hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ if (evs_track_relations_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_relations_hash =
+ hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ need_load = !LWLockHeldByMe(lock);
+ if (need_load)
+ LWLockAcquire(lock, LW_SHARED);
+ PG_TRY();
+ {
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+ }
+ PG_FINALLY();
+ {
+ if (need_load)
+ LWLockRelease(lock);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Load track lists from disk into the backend-local hashes.
+ *
+ * Caller must hold evs_track_lock at least in shared mode, since the file
+ * may be concurrently rewritten by another backend.
+ */
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[MAXPGPATH];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open track file \"%s\": %m", path)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ size_t len = strlen(buf);
+
+ /* Reject unterminated lines (longer than buffer) as corruption. */
+ if (len > 0 && buf[len - 1] != '\n' && !feof(fp))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("line too long in track file \"%s\"", path)));
+
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+
+ if (ferror(fp))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read track file \"%s\": %m", path)));
+ }
+ PG_FINALLY();
+ {
+ FreeFile(fp);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Atomically rewrite the track file. Caller must hold evs_track_lock
+ * in exclusive mode.
+ */
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+ bool failed = false;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s.tmp", path);
+
+ fp = AllocateFile(tmppath, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create track file \"%s\": %m", tmppath)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ if (fputs("[databases]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ if (fprintf(fp, "%u\n", *entry) < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fputs("[relations]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ int rc;
+
+ if (OidIsValid(rel_entry->dboid))
+ rc = fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ rc = fprintf(fp, "0 %u\n", rel_entry->reloid);
+ if (rc < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fflush(fp) != 0)
+ failed = true;
+
+ if (!failed)
+ {
+ int fd = fileno(fp);
+
+ if (fd >= 0 && pg_fsync(fd) != 0)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not fsync track file \"%s\": %m",
+ tmppath)));
+ }
+ }
+ PG_CATCH();
+ {
+ FreeFile(fp);
+ (void) unlink(tmppath);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (FreeFile(fp) != 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close track file \"%s\": %m", tmppath)));
+ failed = true;
+ }
+
+ if (failed)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write track file \"%s\": %m", tmppath)));
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ return;
+ }
+
+ if (durable_rename(tmppath, path, LOG) != 0)
+ {
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ }
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+/*
+ * Mutating track-list entry points: require server-wide privilege, since
+ * the underlying lists steer tracking for every backend.
+ */
+static void
+evs_require_track_privilege(const char *funcname)
+{
+ if (!superuser() && !has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS))
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for function %s", funcname),
+ errhint("Only superusers and members of pg_read_all_stats "
+ "may change the vacuum statistics track list.")));
+}
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base PG_USED_FOR_ASSERTS_ONLY = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid,
+ InvalidOid, NULL);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index ebb7f83d8c5..d7dc0fd07f0 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index b9b03654aad..2a38f9042bb 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 25a85082759..85d721467c0 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v39-0001-Track-table-VM-stability.patch (21.7K, 3-v39-0001-Track-table-VM-stability.patch)
download | inline diff:
From 19f5a39f7e97d3fc2d18415ba2c51ffcd3b32f49 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH 1/3] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
[text/plain] v39-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch (25.0K, 4-v39-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch)
download | inline diff:
From 3a5e0bd82578d1fea63d6bda229dc4d0b224684e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add infrastructure inside lazy vacuum to gather extended per-vacuum
metrics and expose them to extensions via a new hook. Core itself
does not persist these metrics — that is the job of an extension
(see ext_vacuum_statistics).
Statistics are gathered separately for tables and indexes according
to vacuum phases. The ExtVacReport union and type field distinguish
PGSTAT_EXTVAC_TABLE vs PGSTAT_EXTVAC_INDEX. Heap vacuum stats are
sent to the cumulative statistics system after vacuum has processed
the indexes. Database vacuum statistics aggregate per-table and
per-index statistics within the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
set_report_vacuum_hook (set_report_vacuum_hook_type) -- called
once per vacuumed relation/index with a PgStat_VacuumRelationCounts
payload tagged by ExtVacReportType (PGSTAT_EXTVAC_TABLE / _INDEX /
_DB / _INVALID).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 234 ++++++++++++++++++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 12 +
src/backend/utils/activity/pgstat_relation.c | 24 ++
src/include/commands/vacuum.h | 29 +++
src/include/pgstat.h | 69 ++++++
6 files changed, 367 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 39395aed0d5..e4d4c93d641 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -283,6 +283,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -410,6 +412,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -485,6 +496,166 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -643,7 +814,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_vacuum_min_duration >= 0));
@@ -660,6 +834,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -687,7 +864,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -803,6 +982,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -985,11 +1167,26 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+
pgstat_progress_end_command();
if (instrument)
@@ -2903,6 +3100,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3015,7 +3213,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3033,6 +3235,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3041,6 +3244,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3065,7 +3276,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3084,12 +3299,21 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 99d0db82ed7..a7fb73173f5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2561,6 +2564,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 41cefcfde54..200f12a2d1b 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1076,6 +1076,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -1084,6 +1086,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -1112,6 +1116,13 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1276,6 +1287,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 92e1f60a080..226d7aa06d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -272,6 +272,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 956d9cea36d..a925f7da992 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -21,9 +21,11 @@
#include "catalog/pg_class.h"
#include "catalog/pg_statistic.h"
#include "catalog/pg_type.h"
+#include "executor/instrument.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -354,6 +356,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7db36cf8add..8d934973dc1 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,64 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -703,6 +761,17 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
[text/plain] v39-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch (145.2K, 5-v39-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch)
download | inline diff:
From 3011a3cfd9ee3d6e4d1c5a12e3d9984f6b6a194e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 28 Apr 2026 03:43:29 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics: extension for extended vacuum
statistics
Introduce a new extension that collects extended per-vacuum
metrics via set_report_vacuum_hook and stores them through pgstat's
custom statistics infrastructure.
Tracking scope is controlled by GUCs:
* vacuum_statistics.enabled -- master switch
* vacuum_statistics.object_types -- databases / relations / all
* vacuum_statistics.track_relations -- system / user / all
* vacuum_statistics.track_{databases,relations}_from_list
-- restrict tracking to objects registered via
add_track_database() / add_track_relation();
removal via remove_track_*() and OAT_DROP hook
* vacuum_statistics.collect -- buffers / wal /
general / timing / all, consulted by ACCUM_IF() to skip
unwanted categories at run time
add_track_* / remove_track_* require superuser or pg_read_all_stats.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 ++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 272 ++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 ++++
.../t/054_vacuum_extending_gucs_test.pl | 279 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1387 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 ++++++
doc/src/sgml/filelist.sgml | 1 +
18 files changed, 3909 insertions(+)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7d91fe77db3..3140f2bf844 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..aa3a9ec9699
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,272 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Track-list mutation requires superuser or pg_read_all_stats; hide the
+-- functions from PUBLIC so the error is also produced for ordinary users
+-- before the C-level privilege check runs.
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) TO pg_read_all_stats;
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT STABLE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..4f8f025c63e
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_frozen_page_marks_cleared(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_visible_page_marks_cleared(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_frozen_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_visible_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..a195249842b
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,279 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra),
+ # Make pending stats visible to subsequent sessions without sleeping.
+ "SELECT pg_stat_force_next_flush();");
+ $node->safe_psql($db, $sql);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session. Assert not only that the
+ # row count is zero, but that the specific counters remain zero: a stray
+ # row with zero counters would otherwise pass a bare COUNT(*)=0 check.
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no rows when disabled');
+
+ my $sums = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_dirtied), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($sums, '0', 'no counters accumulated when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 6: vacuum_statistics.collect - per-category gating
+#
+# With collect='wal' only wal_* counters must advance; buffer, timing, and
+# general categories must stay at zero. With collect='buffers' the inverse
+# holds. Unknown tokens must be rejected by the check-hook.
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.collect' => sub {
+ # wal-only: WAL counters should accumulate, buffers/timing/general should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'wal'"],
+ modify => 1,
+ });
+
+ my $wal = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal, 't', "collect='wal': wal_records accumulated");
+
+ my $other = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0)
+ + COALESCE(SUM(total_time), 0)
+ + COALESCE(SUM(tuples_deleted), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($other, '0',
+ "collect='wal': buffer/timing/general counters not accumulated");
+
+ # buffers-only: buffer counters should advance, WAL should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'buffers'"],
+ modify => 1,
+ });
+
+ my $buf = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($buf, 't', "collect='buffers': buffer counters accumulated");
+
+ my $wal_off = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal_off, '0',
+ "collect='buffers': WAL counters not accumulated");
+
+ # Unknown category must be rejected by the check-hook.
+ my ($ret, $stdout, $stderr) = $node->psql($dbname,
+ "SET vacuum_statistics.collect = 'nope'");
+ isnt($ret, 0, "collect='nope': rejected by check-hook");
+ like($stderr, qr/Unrecognized category "nope"/,
+ "collect='nope': errdetail names the offending token");
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..144b9bcb814
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1387 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_authid.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+#include "utils/tuplestore.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/*
+ * Bit flags for evs_collect_mask. Each category groups counters that can be
+ * accumulated (or skipped) together, letting users reduce overhead at run
+ * time by turning off categories they don't need.
+ */
+#define EVS_COLLECT_BUFFERS 0x1 /* blks_*, blk_*_time */
+#define EVS_COLLECT_WAL 0x2 /* wal_records, wal_fpi, wal_bytes */
+#define EVS_COLLECT_GENERAL 0x4 /* tuples_deleted, pages_*, vm_*,
+ * wraparound_failsafe_count,
+ * interrupts_count */
+#define EVS_COLLECT_TIMING 0x8 /* delay_time, total_time */
+#define EVS_COLLECT_ALL (EVS_COLLECT_BUFFERS | EVS_COLLECT_WAL | \
+ EVS_COLLECT_GENERAL | EVS_COLLECT_TIMING)
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+static char *evs_collect = "all"; /* categories to collect */
+static int evs_collect_mask = EVS_COLLECT_ALL;
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+static shmem_request_hook_type prev_shmem_request_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+static void evs_shmem_request(void);
+
+/* Hash tables for track_databases and track_relations_list (backend-local) */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+/*
+ * Named LWLock tranche protecting the on-disk track file and serializing
+ * backend-local reloads/saves across concurrent backends.
+ */
+#define EVS_TRACK_TRANCHE_NAME "ext_vacuum_statistics_track"
+static LWLock *evs_track_lock = NULL;
+
+static inline LWLock *
+evs_get_track_lock(void)
+{
+ if (evs_track_lock == NULL)
+ evs_track_lock = &GetNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME)->lock;
+ return evs_track_lock;
+}
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/*
+ * Accumulate a single counter only if its category is enabled in
+ * evs_collect_mask. Parentheses around every argument: the macro is invoked
+ * from expression contexts and with expressions as the destination pointer.
+ */
+#define ACCUM_IF(dst, src, field, cat) \
+ do { \
+ if ((evs_collect_mask) & (cat)) \
+ ((dst))->field += ((src))->field; \
+ } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_dirtied, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_written, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_fetched, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_read_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_write_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, delay_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, total_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, wal_records, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_fpi, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_bytes, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wraparound_failsafe_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, interrupts_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, tuples_deleted, EVS_COLLECT_GENERAL);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/*
+ * GUC check hooks: validate the string and compute the bitmask into *extra.
+ * Rejecting unknown values here prevents silent fall-through to "all".
+ */
+static bool
+evs_track_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "databases") == 0)
+ *bits = EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "relations") == 0)
+ *bits = EVS_TRACK_RELATIONS;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"databases\", \"relations\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ evs_track_bits = *((int *) extra);
+}
+
+static bool
+evs_track_relations_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+ else if (strcmp(*newval, "system") == 0)
+ *bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(*newval, "user") == 0)
+ *bits = EVS_FILTER_USER;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"system\", \"user\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ evs_track_relations_bits = *((int *) extra);
+}
+
+/*
+ * Check hook for vacuum_statistics.collect.
+ *
+ * Accepts a comma- or whitespace-separated list of category names
+ * (buffers, wal, general, timing) or the shorthand "all". Computes the
+ * matching bitmask once and stashes it in *extra; the assign hook just
+ * copies it into evs_collect_mask. Unknown tokens are rejected so the
+ * setting cannot silently collapse to the "all" default.
+ */
+static bool
+evs_collect_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *mask;
+ char *copy;
+ char *p;
+ char *tok;
+ int accum = 0;
+ bool saw_all = false;
+
+ if (*newval == NULL)
+ return false;
+
+ mask = (int *) guc_malloc(LOG, sizeof(int));
+ if (!mask)
+ return false;
+
+ /* Empty string means "all", matching the default behavior. */
+ if ((*newval)[0] == '\0')
+ {
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+ }
+
+ copy = pstrdup(*newval);
+ for (p = copy; (tok = strtok(p, " \t,")) != NULL; p = NULL)
+ {
+ if (pg_strcasecmp(tok, "all") == 0)
+ saw_all = true;
+ else if (pg_strcasecmp(tok, "buffers") == 0)
+ accum |= EVS_COLLECT_BUFFERS;
+ else if (pg_strcasecmp(tok, "wal") == 0)
+ accum |= EVS_COLLECT_WAL;
+ else if (pg_strcasecmp(tok, "general") == 0)
+ accum |= EVS_COLLECT_GENERAL;
+ else if (pg_strcasecmp(tok, "timing") == 0)
+ accum |= EVS_COLLECT_TIMING;
+ else
+ {
+ /*
+ * GUC_check_errdetail formats the message immediately, but tok
+ * points into copy; emit the detail first, then free the
+ * scratch buffer so the formatted string is already stashed in
+ * GUC_check_errdetail_string.
+ */
+ GUC_check_errdetail("Unrecognized category \"%s\" in vacuum_statistics.collect; "
+ "allowed values are \"all\", \"buffers\", \"wal\", \"general\", \"timing\".",
+ tok);
+ pfree(copy);
+ guc_free(mask);
+ return false;
+ }
+ }
+ pfree(copy);
+
+ *mask = saw_all ? EVS_COLLECT_ALL : accum;
+ if (*mask == 0)
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+}
+
+static void
+evs_collect_assign_hook(const char *newval, void *extra)
+{
+ evs_collect_mask = *((int *) extra);
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0,
+ evs_track_check_hook,
+ evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0,
+ evs_track_relations_check_hook,
+ evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.collect",
+ "Statistics categories to collect.",
+ "Comma- or whitespace-separated list of: "
+ "\"buffers\", \"wal\", \"general\", \"timing\"; "
+ "or \"all\" for every category (default).",
+ &evs_collect, "all",
+ PGC_SUSET, 0,
+ evs_collect_check_hook,
+ evs_collect_assign_hook, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = evs_shmem_request;
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+static void
+evs_shmem_request(void)
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
+ RequestNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME, 1);
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ LWLock *lock = evs_get_track_lock();
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ LWLock *lock = evs_get_track_lock();
+ bool found;
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+/*
+ * Initialize the backend-local tracking hashes and load their contents
+ * from the on-disk file.
+ *
+ * The hashes are per-backend, so no lock is needed to protect them from
+ * other processes; however, another backend may be concurrently rewriting
+ * the track file, so we take a shared lock for the file read.
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+ LWLock *lock;
+ bool need_load;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ lock = evs_get_track_lock();
+
+ if (evs_track_databases_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_databases_hash =
+ hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ if (evs_track_relations_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_relations_hash =
+ hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ need_load = !LWLockHeldByMe(lock);
+ if (need_load)
+ LWLockAcquire(lock, LW_SHARED);
+ PG_TRY();
+ {
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+ }
+ PG_FINALLY();
+ {
+ if (need_load)
+ LWLockRelease(lock);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Load track lists from disk into the backend-local hashes.
+ *
+ * Caller must hold evs_track_lock at least in shared mode, since the file
+ * may be concurrently rewritten by another backend.
+ */
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[MAXPGPATH];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open track file \"%s\": %m", path)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ size_t len = strlen(buf);
+
+ /* Reject unterminated lines (longer than buffer) as corruption. */
+ if (len > 0 && buf[len - 1] != '\n' && !feof(fp))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("line too long in track file \"%s\"", path)));
+
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+
+ if (ferror(fp))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read track file \"%s\": %m", path)));
+ }
+ PG_FINALLY();
+ {
+ FreeFile(fp);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Atomically rewrite the track file. Caller must hold evs_track_lock
+ * in exclusive mode.
+ */
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+ bool failed = false;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s.tmp", path);
+
+ fp = AllocateFile(tmppath, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create track file \"%s\": %m", tmppath)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ if (fputs("[databases]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ if (fprintf(fp, "%u\n", *entry) < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fputs("[relations]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ int rc;
+
+ if (OidIsValid(rel_entry->dboid))
+ rc = fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ rc = fprintf(fp, "0 %u\n", rel_entry->reloid);
+ if (rc < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fflush(fp) != 0)
+ failed = true;
+
+ if (!failed)
+ {
+ int fd = fileno(fp);
+
+ if (fd >= 0 && pg_fsync(fd) != 0)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not fsync track file \"%s\": %m",
+ tmppath)));
+ }
+ }
+ PG_CATCH();
+ {
+ FreeFile(fp);
+ (void) unlink(tmppath);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (FreeFile(fp) != 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close track file \"%s\": %m", tmppath)));
+ failed = true;
+ }
+
+ if (failed)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write track file \"%s\": %m", tmppath)));
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ return;
+ }
+
+ if (durable_rename(tmppath, path, LOG) != 0)
+ {
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ }
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+/*
+ * Mutating track-list entry points: require server-wide privilege, since
+ * the underlying lists steer tracking for every backend.
+ */
+static void
+evs_require_track_privilege(const char *funcname)
+{
+ if (!superuser() && !has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS))
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for function %s", funcname),
+ errhint("Only superusers and members of pg_read_all_stats "
+ "may change the vacuum statistics track list.")));
+}
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base PG_USED_FOR_ASSERTS_ONLY = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid,
+ InvalidOid, NULL);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index ebb7f83d8c5..d7dc0fd07f0 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index b9b03654aad..2a38f9042bb 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 25a85082759..85d721467c0 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 77+ messages in thread
end of thread, other threads:[~2026-04-28 05:28 UTC | newest]
Thread overview: 77+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-08-15 08:49 ` Alena Rybakina <[email protected]>
2024-08-15 09:50 ` Ilia Evdokimov <[email protected]>
2024-08-16 11:12 ` jian he <[email protected]>
2024-08-19 09:32 ` jian he <[email protected]>
2024-08-19 16:28 ` Ilia Evdokimov <[email protected]>
2024-08-20 22:39 ` Alena Rybakina <[email protected]>
2024-08-23 01:07 ` Alexander Korotkov <[email protected]>
2024-08-25 15:59 ` Alena Rybakina <[email protected]>
2024-08-26 11:55 ` Alena Rybakina <[email protected]>
2024-09-04 17:23 ` Alena Rybakina <[email protected]>
2024-09-05 12:47 ` jian he <[email protected]>
2024-09-05 21:00 ` Alena Rybakina <[email protected]>
2024-09-27 18:15 ` Masahiko Sawada <[email protected]>
2024-09-27 19:19 ` Melanie Plageman <[email protected]>
2024-09-27 20:13 ` Masahiko Sawada <[email protected]>
2024-09-28 21:22 ` Alena Rybakina <[email protected]>
2024-09-27 19:25 ` Andrei Zubkov <[email protected]>
2024-10-08 16:18 ` Alena Rybakina <[email protected]>
2024-10-16 10:31 ` Ilia Evdokimov <[email protected]>
2024-10-16 11:01 ` Alena Rybakina <[email protected]>
2024-10-22 19:30 ` Alena Rybakina <[email protected]>
2024-11-07 14:49 ` Ilia Evdokimov <[email protected]>
2024-10-16 11:17 ` Andrei Zubkov <[email protected]>
2024-10-28 13:40 ` Alexander Korotkov <[email protected]>
2024-10-29 11:02 ` Alena Rybakina <[email protected]>
2024-11-02 12:24 ` Alena Rybakina <[email protected]>
2025-04-22 18:23 ` Andrei Lepikhov <[email protected]>
2025-05-09 12:31 ` Alena Rybakina <[email protected]>
2024-08-20 22:37 ` Alena Rybakina <[email protected]>
2024-08-22 02:47 ` jian he <[email protected]>
2024-08-22 04:29 ` Kirill Reshke <[email protected]>
2024-08-25 16:12 ` Alena Rybakina <[email protected]>
2024-08-25 16:06 ` Alena Rybakina <[email protected]>
2024-08-20 22:35 ` Alena Rybakina <[email protected]>
2025-03-10 09:13 ` Ilia Evdokimov <[email protected]>
2025-03-12 19:41 ` Alena Rybakina <[email protected]>
2025-03-12 22:15 ` Jim Nasby <[email protected]>
2025-03-13 06:42 ` Bertrand Drouvot <[email protected]>
2025-03-21 19:42 ` Alena Rybakina <[email protected]>
2025-03-21 19:46 ` Alena Rybakina <[email protected]>
2025-03-24 23:02 ` Jim Nasby <[email protected]>
2025-03-25 06:12 ` Alena Rybakina <[email protected]>
2025-05-09 12:03 ` Alena Rybakina <[email protected]>
2025-05-12 12:30 ` Amit Kapila <[email protected]>
2025-05-13 09:49 ` Alena Rybakina <[email protected]>
2025-05-14 08:55 ` Amit Kapila <[email protected]>
2025-06-02 16:25 ` Alexander Korotkov <[email protected]>
2025-06-02 16:50 ` Alena Rybakina <[email protected]>
2025-09-04 15:49 ` Alena Rybakina <[email protected]>
2025-09-04 16:18 ` Alena Rybakina <[email protected]>
2025-06-02 19:56 ` Alena Rybakina <[email protected]>
2025-06-03 12:27 ` Alena Rybakina <[email protected]>
2025-09-01 19:13 ` Alena Rybakina <[email protected]>
2025-09-15 20:46 ` Ilia Evdokimov <[email protected]>
2025-09-25 13:53 ` Alena Rybakina <[email protected]>
2025-12-20 23:36 ` Alena Rybakina <[email protected]>
2026-03-09 15:46 ` Alena Rybakina <[email protected]>
2026-03-12 12:02 ` Andrei Lepikhov <[email protected]>
2026-03-12 13:28 ` Andrei Lepikhov <[email protected]>
2026-03-12 18:10 ` Alena Rybakina <[email protected]>
2026-03-13 13:04 ` Alena Rybakina <[email protected]>
2026-03-16 08:45 ` Andrei Lepikhov <[email protected]>
2026-03-16 12:11 ` Alena Rybakina <[email protected]>
2026-03-16 14:27 ` Andrey Borodin <[email protected]>
2026-03-30 06:13 ` Alena Rybakina <[email protected]>
2026-04-14 11:10 ` Alena Rybakina <[email protected]>
2026-04-28 02:16 ` Alena Rybakina <[email protected]>
2026-04-28 05:28 ` Alena Rybakina <[email protected]>
2026-03-16 12:07 ` Alena Rybakina <[email protected]>
2026-03-17 15:27 ` Andrei Zubkov <[email protected]>
2026-03-18 14:19 ` Andrei Zubkov <[email protected]>
2025-09-25 00:03 ` Bharath Rupireddy <[email protected]>
2025-09-25 15:10 ` Alena Rybakina <[email protected]>
2025-03-10 13:33 ` Kirill Reshke <[email protected]>
2025-03-12 19:36 ` Alena Rybakina <[email protected]>
2025-03-17 06:42 ` vignesh C <[email protected]>
2025-03-18 05:57 ` Alena Rybakina <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox