public inbox for [email protected]
help / color / mirror / Atom feedRe: Vacuum statistics
46+ messages / 11 participants
[nested] [flat]
* Re: Vacuum statistics
@ 2024-10-28 19:07 Alexander Korotkov <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Alexander Korotkov @ 2024-10-28 19:07 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; Andrei Zubkov <[email protected]>; jian he <[email protected]>; Ilia Evdokimov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
On Sun, Sep 29, 2024 at 12:22 AM Alena Rybakina
<[email protected]> wrote:
> Hi! Thank you for your interesting for this patch!
>
> I took a very brief look at this and was wondering if it was worth
> having a way to make the per-table vacuum statistics opt-in (like a
> table storage parameter) in order to decrease the shared memory
> footprint of storing the stats.
>
> I'm not sure how users can select tables that enable vacuum statistics
> as I think they basically want to have statistics for all tables, but
> I see your point. Since the size of PgStat_TableCounts approximately
> tripled by this patch (112 bytes to 320 bytes), it might be worth
> considering ways to reduce the number of entries or reducing the size
> of vacuum statistics.
>
> The main purpose of these statistics is to see abnormal behavior of vacuum in relation to a table or the database as a whole.
>
> For example, there may be a situation where vacuum has started to run more often and spends a lot of resources on processing a certain index, but the size of the index does not change significantly. Moreover, the table in which this index is located can be much smaller in size. This may be because the index is bloated and needs to be reindexed.
>
> This is exactly what vacuum statistics can show - we will see that compared to other objects, vacuum processed more blocks and spent more time on this index.
>
> Perhaps the vacuum parameters for the index should be set more aggressively to avoid this in the future.
>
> I suppose that if we turn off statistics collection for a certain object, we can miss it. In addition, the user may not enable the parameter for the object in time, because he will forget about it.
I agree with this point. Additionally, in order to benefit from
gatherting vacuum statistics only for some relations in terms of
space, we need to handle variable-size stat entries. That would
greatly increase the complexity.
> As for the second option, now I cannot say which statistics can be removed, to be honest. So far, they all seem necessary.
Yes, but as Masahiko-san pointed out, PgStat_TableCounts is almost
tripled in space. That a huge change from having no statistics on
vacuum to have it in much more detail than everything else we
currently have. I think the feasible way might be to introduce some
most demanded statistics first then see how it goes.
------
Regards,
Alexander Korotkov
Supabase
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-10-28 21:03 Jim Nasby <[email protected]>
parent: Alexander Korotkov <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Jim Nasby @ 2024-10-28 21:03 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Alena Rybakina <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; Andrei Zubkov <[email protected]>; jian he <[email protected]>; Ilia Evdokimov <[email protected]>; Alena Rybakina <[email protected]>; pgsql-hackers; [email protected]
> On Oct 28, 2024, at 2:07 PM, Alexander Korotkov <[email protected]> wrote:
>
>> I suppose that if we turn off statistics collection for a certain object, we can miss it. In addition, the user may not enable the parameter for the object in time, because he will forget about it.
>
> I agree with this point. Additionally, in order to benefit from
> gatherting vacuum statistics only for some relations in terms of
> space, we need to handle variable-size stat entries. That would
> greatly increase the complexity.
Could vacuum stats be treated as a separate category instead of adding it to PgStat_TableCounts?
>> As for the second option, now I cannot say which statistics can be removed, to be honest. So far, they all seem necessary.
>
> Yes, but as Masahiko-san pointed out, PgStat_TableCounts is almost
> tripled in space. That a huge change from having no statistics on
> vacuum to have it in much more detail than everything else we
> currently have. I think the feasible way might be to introduce some
> most demanded statistics first then see how it goes.
Looking at the stats I do think the WAL stats are probably not helpful. First, there’s nothing users can do to tune how much WAL is generated by vacuum. Second, this introduces the risk of users saying “Wow, vacuum is creating a lot of WAL! I’m going to turn it down!”, which is most likely to make matters worse. There’s already a lot of stuff that goes into WAL without any detailed logging; if we ever wanted to provide a comprehensive view of what data is in WAL that should be handled separately.
The rest of the stats all look important. In fact, I think there’s even more stats that could be included (such as all frozen/visible pages skipped) - even more reason to look at having separate controls for tracking vacuum stats. There’s also an argument to be made for tracking autovac separately from manual vacuum. So long-term we might want to look at other ways to handle these stats, not only because of the large number of stats, but because they would be updated very infrequently compared to other stats counters. Ironically, the old stats system would probably have been more than sufficient for these stats. Tracking them in a real table might also be an option.
Is there a reason some fields are omitted from pg_stat_vacuum_database? While some stats are certainly more interesting at the per-relation level, I can’t really think of any that don’t make sense at the database level as well.
Looking at the per table/index stats, I strongly dislike the use of the term “delete” - it is a recipe for confusion with row deletion.. A much better term is “remove” or “removed”. I realize the term “delete” is used in places in vacuum logging, but IMO we should fix that as well instead of doubling-down on it.
I think “interrupts” is also a very confusing name - those fields should just be called “errors”.
I realize “relname” is being used for consistency with pg_stat_all_(tables|indexes), but I’m not sure it makes sense to double-down on that. Especially in pg_stat_vacuum_indexes, where it’s not completely clear whether relname is referring to the table or the index. I’m also inclined to say that the name of the table should be included in pg_stat_vacuum_indexes.
For all the views the docs should clarify that total_blks_written means blocks written by vacuum, as opposed to the background writer. Similarly they should clarify the difference between rel_blks_(read|hit) and total_blks_(read|hit). In the case of pg_stat_vacuum_indexes it’d be better if rel_blks_(read|hit) were called index_blks_(read|hit). Although… if total_blks_* is actually the count across the table and all the indexes I don’t know that we even need that counter. I realize that not ever vacuum even looks at the indexes, but if we’re going to go into that level of detail then we would (at minimum) need to count the number of times a vacuum completely skipped scanning the indexes.
Having rev_all_(frozen|visible)_pages in the same view as vacuum stats will confuse users into thinking that vacuum is clearing the bits. Those fields really belong in pg_stat_all_tables.
Sadly index_vacuum_count is may not useful at all at present. At minimum you’d need to know the number of times vacuum had run in total. I realize that’s in pg_stat_all_tables, but that doesn’t help if vacuum stats are tracked or reset separately. At minimum the docs should mention them. They also need to clarify if index_vacuum_count is incremented per-index or per-pass (hopefully the later). Assuming it’s per-pass, a better name for the field would be index_vacuum_passes, index_passes, index_pass_count, or similar. But even with that we still need a counter for the number of vacuums where index processing was skipped.
Other items
First, thanks to everyone that’s put work into this patch - it’s a big step forward. I certainly don’t want the perfect to be the enemy of the good, but since the size of these stats entries has already come up as a concern I want to consider use cases that would still not be covered by this patch. I’m not suggesting these need to be added now, but IMHO they’re logical next steps (that would also mean more counters). The cases below would probably mean at least doubling the number of vacuum-related counters, at least at the table level.
First, there’s still gaps in trying to track HOT; most notably a counter for how many updates would never be HOT eligible because they modify indexes. pg_stat_all_tables.n_tup_newpage_upd is really limited without that info.
There should also be stats about unused line pointers - in degenerate cases the lp array can consume a significant portion of heap storage.
Monitoring bloat would be a lot more accurate if vacuum reported total tuple length for each run along with the total number of tuples it looked at. Having that info would make it trivial to calculate average tuple size, which could then be applied to reltuples and relpages to calculate how much space would being lost to bloat.
Autovacuum will self-terminate if it would block another process (unless it’s an aggressive vacuum) - that’s definitely something that should be tracked. Not just the number of times that happens, but also stats about how much work was lost because of this.
Shrinking a relation (what vacuum calls truncation, which is very confusing with the truncate command) is a rather complex process that currently has no visibility.
Tuning vacuum_freeze_min_age (and the MXID variant) is rather complicated. We maybe have enough stats on whether it could be set lower, but there’s no visibility on how the settings affect how often vacuum decides to be aggressive. At minimum, we should have stats on when vacuum is aggressive, especially since it significantly changes the behavior of autovac.
I saw someone else already mentioned tuning vacuum memory usage, but I’ll mention it again. Even if the issues with index_vacuum_count are fixed that still only tells you if you have a problem; it doesn’t give you a great idea of how much more memory you need. The best you can do is assuming you need (number of passes - 1) * current memory.
Speaking of which… there should be stats on any time vacuum decided on it’s own to skip index processing due to wraparound proximity.
I’m sure there’s some other use cases that I’m not thinking of.
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-10-29 12:40 Andrei Zubkov <[email protected]>
parent: Jim Nasby <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Andrei Zubkov @ 2024-10-29 12:40 UTC (permalink / raw)
To: Jim Nasby <[email protected]>; Alexander Korotkov <[email protected]>; +Cc: Alena Rybakina <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
Hi,
Thanks for your attention to our patch!
On Mon, 2024-10-28 at 16:03 -0500, Jim Nasby wrote:
> > Yes, but as Masahiko-san pointed out, PgStat_TableCounts is almost
> > tripled in space. That a huge change from having no statistics on
> > vacuum to have it in much more detail than everything else we
> > currently have. I think the feasible way might be to introduce
> > some
> > most demanded statistics first then see how it goes.
>
> Looking at the stats I do think the WAL stats are probably not
> helpful. First, there’s nothing users can do to tune how much WAL is
> generated by vacuum. Second, this introduces the risk of users saying
> “Wow, vacuum is creating a lot of WAL! I’m going to turn it down!”,
> which is most likely to make matters worse. There’s already a lot of
> stuff that goes into WAL without any detailed logging; if we ever
> wanted to provide a comprehensive view of what data is in WAL that
> should be handled separately.
Yes, there is nothing we can directly do with WAL generated by vacuum,
but WAL generation is the part of vacuum work, and it will indirectly
affected by the changes of vacuum settings. So, WAL statistics is one
more dimension of vacuum workload. Also WAL stat is universal metric
which is measured cluster-wide and on the statement-level with
pg_stat_statements. Vacuum WAL counters will explain the part of
difference between those metrics. Besides vacuum WAL counters can be
used to locate abnormal vacuum behavior caused by a bug or the data
corruption. I think if the DBA is smart enough to look at vacuum WAL
generated stats and to understand what it means, the decision to
disable the autovacuum due to its WAL generation is unlikely.
Anyway I think some stats can be excluded to save some memory. The
first candidates are the system_time and user_time fields. Those are
very valuable, but are measured by the rusage stats, which won't be
available on all platforms. I think total_time and delay_time would be
sufficient.
The second is the interrupts field. It is needed for monitoring to know
do we have them or not, so tracking them on the database level will do
the trick. Interrupt is quite rare event, so once the monitoring system
will catch one the DBA can go to the server log for the details.
It seems there is another way. If the vacuum stats doesn't seems to be
mandatory in all systems, maybe we should add some hooks to the vacuum
so that vacuum statistics tracking can be done in an extension. I don't
think it is a good idea, because vacuum stats seems to me as mandatory
as the vacuum process itself.
> Is there a reason some fields are omitted
> from pg_stat_vacuum_database? While some stats are certainly more
> interesting at the per-relation level, I can’t really think of any
> that don’t make sense at the database level as well.
Some of the metrics are table-specific, some index-specific, so we
moved to the database level metrics more or less specific to the whole
database. Can you tell what stats you want to see at the database
level?
> Looking at the per table/index stats, I strongly dislike the use of
> the term “delete” - it is a recipe for confusion with row deletion..
> A much better term is “remove” or “removed”. I realize the term
> “delete” is used in places in vacuum logging, but IMO we should fix
> that as well instead of doubling-down on it.
Yes, this point was discussed in our team, and it seems confusing to me
too. We decided to name it as it is named in the code and to get
feedback from the community. Now we get one. Thank you. Now we should
discuss it and choose the best one. My personal choice is "removed".
> I realize “relname” is being used for consistency with
> pg_stat_all_(tables|indexes), but I’m not sure it makes sense to
> double-down on that. Especially in pg_stat_vacuum_indexes, where it’s
> not completely clear whether relname is referring to the table or the
> index. I’m also inclined to say that the name of the table should be
> included in pg_stat_vacuum_indexes.
Agreed. Table name is needed in the index view.
> For all the views the docs should clarify that total_blks_written
> means blocks written by vacuum, as opposed to the background Ywriter.
We have the "Number of database blocks written by vacuum operations
performed on this table" in the docs now. Do you mean we should
specifically note the vacuum process here?
> Similarly they should clarify the difference between
> rel_blks_(read|hit) and total_blks_(read|hit). In the case of
> pg_stat_vacuum_indexes it’d be better if rel_blks_(read|hit) were
> called index_blks_(read|hit). Although… if total_blks_* is actually
> the count across the table and all the indexes I don’t know that we
> even need that counter. I realize that not ever vacuum even looks at
> the indexes, but if we’re going to go into that level of detail then
> we would (at minimum) need to count the number of times a vacuum
> completely skipped scanning the indexes.
It is not clear to me enough. The stats described just as it is -
rel_blocks_* tracks blocks of the current heap, and total_* is for the
whole database blocks - not just tables and indexes, vacuum do some
work (quite a little) in the catalog and this work is counted here too.
Usually this stat won't be helpful, but maybe we can catch unusual
vacuum behavior using this stat.
> Having rev_all_(frozen|visible)_pages in the same view as vacuum
> stats will confuse users into thinking that vacuum is clearing the
> bits. Those fields really belong in pg_stat_all_tables.
Agreed.
> Sadly index_vacuum_count is may not useful at all at present. At
> minimum you’d need to know the number of times vacuum had run in
> total. I realize that’s in pg_stat_all_tables, but that doesn’t help
> if vacuum stats are tracked or reset separately.
I'm in doubt - is it really possible to reset the vacuum stats
independent of pg_stat_all_tables?
> At minimum the docs should mention them. They also need to clarify
> if index_vacuum_count is incremented per-index or per-pass (hopefully
> the later). Assuming it’s per-pass, a better name for the field would
> be index_vacuum_passes, index_passes, index_pass_count, or similar.
> But even with that we still need a counter for the number of vacuums
> where index processing was skipped.
Agreed, the "index_passes" looks good to me, and index processing skip
counter looks good.
> First, there’s still gaps in trying to track HOT; most notably a
> counter for how many updates would never be HOT eligible because they
> modify indexes. pg_stat_all_tables.n_tup_newpage_upd is really
> limited without that info.
Nice catch, I'll think about it. Those are not directly connected to
the vacuum workload but those are important.
> There should also be stats about unused line pointers - in degenerate
> cases the lp array can consume a significant portion of heap storage.
>
> Monitoring bloat would be a lot more accurate if vacuum reported
> total tuple length for each run along with the total number of tuples
> it looked at. Having that info would make it trivial to calculate
> average tuple size, which could then be applied to reltuples and
> relpages to calculate how much space would being lost to bloat.
Yes, bloat tracking is in our plans. Right now it is not clear enough
how to do it in the most reliable and convenient way.
> Autovacuum will self-terminate if it would block another process
> (unless it’s an aggressive vacuum) - that’s definitely something that
> should be tracked. Not just the number of times that happens, but
> also stats about how much work was lost because of this.
Agreed.
> Shrinking a relation (what vacuum calls truncation, which is very
> confusing with the truncate command) is a rather complex process that
> currently has no visibility.
In this patch table truncation can be seen in the "pages_removed" field
of "pg_stat_vacuum_tables" at least as the cumulative number of removed
pages. It is not clear enough, but it is visible.
> Tuning vacuum_freeze_min_age (and the MXID variant) is rather
> complicated. We maybe have enough stats on whether it could be set
> lower, but there’s no visibility on how the settings affect how often
> vacuum decides to be aggressive. At minimum, we should have stats on
> when vacuum is aggressive, especially since it significantly changes
> the behavior of autovac.
When you say "agressive" do you mean the number of times when the
vacuum was processing the table with the FREEZE intention? I think this
is needed too.
> I saw someone else already mentioned tuning vacuum memory usage, but
> I’ll mention it again. Even if the issues with index_vacuum_count are
> fixed that still only tells you if you have a problem; it doesn’t
> give you a great idea of how much more memory you need. The best you
> can do is assuming you need (number of passes - 1) * current memory.
Do you think such approach is insufficient? It seems we do not need
byte-to-byte accuracy here.
> Speaking of which… there should be stats on any time vacuum decided
> on it’s own to skip index processing due to wraparound proximity.
Maybe we should just count the number of times when the vacuum was
started to prevent wraparound?
Jim, thank you for such detailed review of our patch!
--
regards, Andrei Zubkov
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-10-29 22:23 Jim Nasby <[email protected]>
parent: Andrei Zubkov <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Jim Nasby @ 2024-10-29 22:23 UTC (permalink / raw)
To: Andrei Zubkov <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; Alena Rybakina <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
On Oct 29, 2024, at 7:40 AM, Andrei Zubkov <[email protected]> wrote:
>
> Hi,
>
> Thanks for your attention to our patch!
>
> On Mon, 2024-10-28 at 16:03 -0500, Jim Nasby wrote:
>>> Yes, but as Masahiko-san pointed out, PgStat_TableCounts is almost
>>> tripled in space. That a huge change from having no statistics on
>>> vacuum to have it in much more detail than everything else we
>>> currently have. I think the feasible way might be to introduce
>>> some
>>> most demanded statistics first then see how it goes.
>>
>> Looking at the stats I do think the WAL stats are probably not
>> helpful. First, there’s nothing users can do to tune how much WAL is
>> generated by vacuum. Second, this introduces the risk of users saying
>> “Wow, vacuum is creating a lot of WAL! I’m going to turn it down!”,
>> which is most likely to make matters worse. There’s already a lot of
>> stuff that goes into WAL without any detailed logging; if we ever
>> wanted to provide a comprehensive view of what data is in WAL that
>> should be handled separately.
>
> Yes, there is nothing we can directly do with WAL generated by vacuum,
> but WAL generation is the part of vacuum work, and it will indirectly
> affected by the changes of vacuum settings. So, WAL statistics is one
> more dimension of vacuum workload. Also WAL stat is universal metric
> which is measured cluster-wide and on the statement-level with
> pg_stat_statements. Vacuum WAL counters will explain the part of
> difference between those metrics. Besides vacuum WAL counters can be
> used to locate abnormal vacuum behavior caused by a bug or the data
> corruption. I think if the DBA is smart enough to look at vacuum WAL
> generated stats and to understand what it means, the decision to
> disable the autovacuum due to its WAL generation is unlikely.
I’m generally for more stats rather than less - really just a question of how much we’re worried about stats overhead.
> Anyway I think some stats can be excluded to save some memory. The
> first candidates are the system_time and user_time fields. Those are
> very valuable, but are measured by the rusage stats, which won't be
> available on all platforms. I think total_time and delay_time would be
> sufficient.
Yeah, I considered throwing those under the bus. I agree they’re only marginally useful.
> The second is the interrupts field. It is needed for monitoring to know
> do we have them or not, so tracking them on the database level will do
> the trick. Interrupt is quite rare event, so once the monitoring system
> will catch one the DBA can go to the server log for the details.
Just to confirm… by “interrupt” you mean vacuum encountered an error?
> It seems there is another way. If the vacuum stats doesn't seems to be
> mandatory in all systems, maybe we should add some hooks to the vacuum
> so that vacuum statistics tracking can be done in an extension. I don't
> think it is a good idea, because vacuum stats seems to me as mandatory
> as the vacuum process itself.
I’d actually like hooks for all stats, so people can develop different ways of storing/aggregating them. But I agree that’s a separate discussion.
>> Is there a reason some fields are omitted
>> from pg_stat_vacuum_database? While some stats are certainly more
>> interesting at the per-relation level, I can’t really think of any
>> that don’t make sense at the database level as well.
>
> Some of the metrics are table-specific, some index-specific, so we
> moved to the database level metrics more or less specific to the whole
> database. Can you tell what stats you want to see at the database
> level?
Here’s the thing with pg_stat_vacuum_database; it’s the only way to see everything in the whole cluster. So I think the better question is what metrics simply don’t make sense at that level? And I don’t really see any that don’t.
>> For all the views the docs should clarify that total_blks_written
>> means blocks written by vacuum, as opposed to the background Ywriter.
>
> We have the "Number of database blocks written by vacuum operations
> performed on this table" in the docs now. Do you mean we should
> specifically note the vacuum process here?
The reason the stat is confusing is because it doesn’t have the meaning that the name implies. Most people that see this will think it’s actually measuring blocks dirtied, or at least something closer to that. It definitely hides the fact that many of the dirtied blocks could actually be written by the bgwriter. So an improvement to the docs would be “Number of blocks written directly by vacuum or auto vacuum. Blocks that are dirtied by a vacuum process can be written out by another process.”
Which makes me realize… I think vacuum only counts a block as dirtied if it was previously clean? If so the docs for that metric need to clarify that vacuum might modify a block but not count it as having been dirtied.
>> Similarly they should clarify the difference between
>> rel_blks_(read|hit) and total_blks_(read|hit). In the case of
>> pg_stat_vacuum_indexes it’d be better if rel_blks_(read|hit) were
>> called index_blks_(read|hit). Although… if total_blks_* is actually
>> the count across the table and all the indexes I don’t know that we
>> even need that counter. I realize that not ever vacuum even looks at
>> the indexes, but if we’re going to go into that level of detail then
>> we would (at minimum) need to count the number of times a vacuum
>> completely skipped scanning the indexes.
>
> It is not clear to me enough. The stats described just as it is -
> rel_blocks_* tracks blocks of the current heap, and total_* is for the
> whole database blocks - not just tables and indexes, vacuum do some
> work (quite a little) in the catalog and this work is counted here too.
> Usually this stat won't be helpful, but maybe we can catch unusual
> vacuum behavior using this stat.
Ok, so this just needs to be clarified in the docs by explicitly stating what is and isn’t part of the metric. It would also be better not to use the term “rel” since most people don’t immediately know what that means. So, table_blks_(read|hit) or index_blks_(read|hit).
Also, “total” is still not clear to me, at least in the context of pg_stat_vacuum_indexes. Is that different from pg_stat_vacuum_tables.total_blks_*? If so, how? If it’s the same then IMO it should just be removed from pg_stat_vacuum_indexes.
>> Sadly index_vacuum_count is may not useful at all at present. At
>> minimum you’d need to know the number of times vacuum had run in
>> total. I realize that’s in pg_stat_all_tables, but that doesn’t help
>> if vacuum stats are tracked or reset separately.
>
> I'm in doubt - is it really possible to reset the vacuum stats
> independent of pg_stat_all_tables?
Most stats can be independently reset, so I was thinking these wouldn’t be an exception. If that’s not the case then I think the docs need to mention pg_stat_all_tables.(auto)vacuum_count, since it’s in a completely different view. Or better yet, include the vacuum/analyze related stats that are in pg_stat_all_tables in pg_stat_vacuum_tables.
BTW, have you thought about what stats should be added for ANALYZE? That’s obviously not as critical as vacuum, but maybe worth considering as part of this...
>> First, there’s still gaps in trying to track HOT; most notably a
>> counter for how many updates would never be HOT eligible because they
>> modify indexes. pg_stat_all_tables.n_tup_newpage_upd is really
>> limited without that info.
>
> Nice catch, I'll think about it. Those are not directly connected to
> the vacuum workload but those are important.
Just to re-iterate: I don’t think this patch has to boil the ocean and try to handle all these extra use cases.
>> There should also be stats about unused line pointers - in degenerate
>> cases the lp array can consume a significant portion of heap storage.
>>
>> Monitoring bloat would be a lot more accurate if vacuum reported
>> total tuple length for each run along with the total number of tuples
>> it looked at. Having that info would make it trivial to calculate
>> average tuple size, which could then be applied to reltuples and
>> relpages to calculate how much space would being lost to bloat.
>
> Yes, bloat tracking is in our plans. Right now it is not clear enough
> how to do it in the most reliable and convenient way.
>
>> Autovacuum will self-terminate if it would block another process
>> (unless it’s an aggressive vacuum) - that’s definitely something that
>> should be tracked. Not just the number of times that happens, but
>> also stats about how much work was lost because of this.
>
> Agreed.
>
>> Shrinking a relation (what vacuum calls truncation, which is very
>> confusing with the truncate command) is a rather complex process that
>> currently has no visibility.
>
> In this patch table truncation can be seen in the "pages_removed" field
> of "pg_stat_vacuum_tables" at least as the cumulative number of removed
> pages. It is not clear enough, but it is visible.
Ahh, good point. I think it’s probably worth adding a counter (to this patch) for how many times vacuum actually decided to do page removal, because it’s (presumably) a pretty rare event. Without that counter it’s very hard to make any sense of the number of pages removed (other than being able to see some were removed, at least once).
>> Tuning vacuum_freeze_min_age (and the MXID variant) is rather
>> complicated. We maybe have enough stats on whether it could be set
>> lower, but there’s no visibility on how the settings affect how often
>> vacuum decides to be aggressive. At minimum, we should have stats on
>> when vacuum is aggressive, especially since it significantly changes
>> the behavior of autovac.
>
> When you say "agressive" do you mean the number of times when the
> vacuum was processing the table with the FREEZE intention? I think this
> is needed too.
Yes. I intentionally use the term “aggressive” (as the code does) to avoid confusion with the FREEZE option (which as I’m sure you know simply forces some GUCs to 0). Further complicating this is that auto vac will report this as “to prevent wraparound”…
In any case… I’m actually leaning towards there should be a complete second set of counters for aggressive vacuums, because of how differently they work. :(
>> I saw someone else already mentioned tuning vacuum memory usage, but
>> I’ll mention it again. Even if the issues with index_vacuum_count are
>> fixed that still only tells you if you have a problem; it doesn’t
>> give you a great idea of how much more memory you need. The best you
>> can do is assuming you need (number of passes - 1) * current memory.
>
> Do you think such approach is insufficient? It seems we do not need
> byte-to-byte accuracy here.
Byte-for-byte, no. But I do wonder if there’s any way to do better than some multiple of what *_work_mem was set to.
And setting that aside, another significant problem is that you can’t actually do anything here without actually knowing what memory setting was used, which is definitely not a given. Off-hand I don’t see anyway this can actually be tuned (at all) with nothing but counters. :(
Definitely out of scope for this patch though :)
>> Speaking of which… there should be stats on any time vacuum decided
>> on it’s own to skip index processing due to wraparound proximity.
>
> Maybe we should just count the number of times when the vacuum was
> started to prevent wraparound?
Unfortunately even that isn’t simple… auto vac and manual vac have different GUCs, and of course there’s the FREEZE option. And then there’s the issue that MXIDs are handled completely separately.
Even ignoring all of that… by default an aggressive vacuum won’t skip indexes. That only happens when you hit vacuum_(multixact_)failsafe_age.
BTW, something I’ve been mulling over is what stats related to cleanup might be tracked at a system level. I’m thinking along the lines of how often heap_prune_page or the index marking code come across a dead tuple they can’t do anything about yet because it’s still visible. While you could track that per-relation, I’m not sure how helpful that actually is since it’s really a long-running transaction problem.
Similarly, it’d be nice if we had stats about how often all of the auto vac workers were occupied; something that’s also global in nature.
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-11-02 12:22 Alena Rybakina <[email protected]>
parent: Jim Nasby <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2024-11-02 12:22 UTC (permalink / raw)
To: Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
On 30.10.2024 01:23, Jim Nasby wrote:
> On Oct 29, 2024, at 7:40 AM, Andrei Zubkov<[email protected]> wrote:
>> Hi,
>>
>> Thanks for your attention to our patch!
>>
>> On Mon, 2024-10-28 at 16:03 -0500, Jim Nasby wrote:
>>>> Yes, but as Masahiko-san pointed out, PgStat_TableCounts is almost
>>>> tripled in space. That a huge change from having no statistics on
>>>> vacuum to have it in much more detail than everything else we
>>>> currently have. I think the feasible way might be to introduce
>>>> some
>>>> most demanded statistics first then see how it goes.
>>> Looking at the stats I do think the WAL stats are probably not
>>> helpful. First, there’s nothing users can do to tune how much WAL is
>>> generated by vacuum. Second, this introduces the risk of users saying
>>> “Wow, vacuum is creating a lot of WAL! I’m going to turn it down!”,
>>> which is most likely to make matters worse. There’s already a lot of
>>> stuff that goes into WAL without any detailed logging; if we ever
>>> wanted to provide a comprehensive view of what data is in WAL that
>>> should be handled separately.
>> Yes, there is nothing we can directly do with WAL generated by vacuum,
>> but WAL generation is the part of vacuum work, and it will indirectly
>> affected by the changes of vacuum settings. So, WAL statistics is one
>> more dimension of vacuum workload. Also WAL stat is universal metric
>> which is measured cluster-wide and on the statement-level with
>> pg_stat_statements. Vacuum WAL counters will explain the part of
>> difference between those metrics. Besides vacuum WAL counters can be
>> used to locate abnormal vacuum behavior caused by a bug or the data
>> corruption. I think if the DBA is smart enough to look at vacuum WAL
>> generated stats and to understand what it means, the decision to
>> disable the autovacuum due to its WAL generation is unlikely.
> I’m generally for more stats rather than less - really just a question of how much we’re worried about stats overhead.
>
>> Anyway I think some stats can be excluded to save some memory. The
>> first candidates are the system_time and user_time fields. Those are
>> very valuable, but are measured by the rusage stats, which won't be
>> available on all platforms. I think total_time and delay_time would be
>> sufficient.
> Yeah, I considered throwing those under the bus. I agree they’re only marginally useful.
>
>> The second is the interrupts field. It is needed for monitoring to know
>> do we have them or not, so tracking them on the database level will do
>> the trick. Interrupt is quite rare event, so once the monitoring system
>> will catch one the DBA can go to the server log for the details.
> Just to confirm… by “interrupt” you mean vacuum encountered an error?
Yes it is.
I updated patches. I excluded system and user time statistics and save
number of interrupts only for database.I removed the ability to get
statistics for all tables, now they can only be obtained for an oid
table [0], as suggested here. I also renamed the statistics from
pg_stat_vacuum_tables to pg_stat_get_vacuum_tables and similarly for
indexes and databases. I noticed that that’s what they’re mostly called.
Ready for discussion.
>>> For all the views the docs should clarify that total_blks_written
>>> means blocks written by vacuum, as opposed to the background Ywriter.
>> We have the "Number of database blocks written by vacuum operations
>> performed on this table" in the docs now. Do you mean we should
>> specifically note the vacuum process here?
> The reason the stat is confusing is because it doesn’t have the meaning that the name implies. Most people that see this will think it’s actually measuring blocks dirtied, or at least something closer to that. It definitely hides the fact that many of the dirtied blocks could actually be written by the bgwriter. So an improvement to the docs would be “Number of blocks written directly by vacuum or auto vacuum. Blocks that are dirtied by a vacuum process can be written out by another process.”
>
> Which makes me realize… I think vacuum only counts a block as dirtied if it was previously clean? If so the docs for that metric need to clarify that vacuum might modify a block but not count it as having been dirtied.
I think this makes sense, but I haven't fixed it in the documentation
yet. I need time to learn this, to be honest. I'll answer later.
>>> Sadly index_vacuum_count is may not useful at all at present. At
>>> minimum you’d need to know the number of times vacuum had run in
>>> total. I realize that’s in pg_stat_all_tables, but that doesn’t help
>>> if vacuum stats are tracked or reset separately.
>> I'm in doubt - is it really possible to reset the vacuum stats
>> independent of pg_stat_all_tables?
> Most stats can be independently reset, so I was thinking these wouldn’t be an exception. If that’s not the case then I think the docs need to mention pg_stat_all_tables.(auto)vacuum_count, since it’s in a completely different view. Or better yet, include the vacuum/analyze related stats that are in pg_stat_all_tables in pg_stat_vacuum_tables.
To be honest, it was obvious to me, but we can mention it.
>>> Autovacuum will self-terminate if it would block another process
>>> (unless it’s an aggressive vacuum) - that’s definitely something that
>>> should be tracked. Not just the number of times that happens, but
>>> also stats about how much work was lost because of this.
>> Agreed.
>>> Tuning vacuum_freeze_min_age (and the MXID variant) is rather
>>> complicated. We maybe have enough stats on whether it could be set
>>> lower, but there’s no visibility on how the settings affect how often
>>> vacuum decides to be aggressive. At minimum, we should have stats on
>>> when vacuum is aggressive, especially since it significantly changes
>>> the behavior of autovac.
>> When you say "agressive" do you mean the number of times when the
>> vacuum was processing the table with the FREEZE intention? I think this
>> is needed too.
> Yes. I intentionally use the term “aggressive” (as the code does) to avoid confusion with the FREEZE option (which as I’m sure you know simply forces some GUCs to 0). Further complicating this is that auto vac will report this as “to prevent wraparound”…
>
> In any case… I’m actually leaning towards there should be a complete second set of counters for aggressive vacuums, because of how differently they work. :(
>>> Speaking of which… there should be stats on any time vacuum decided
>>> on it’s own to skip index processing due to wraparound proximity.
>> Maybe we should just count the number of times when the vacuum was
>> started to prevent wraparound?
> Unfortunately even that isn’t simple… auto vac and manual vac have different GUCs, and of course there’s the FREEZE option. And then there’s the issue that MXIDs are handled completely separately.
>
> Even ignoring all of that… by default an aggressive vacuum won’t skip indexes. That only happens when you hit vacuum_(multixact_)failsafe_age.
>
> BTW, something I’ve been mulling over is what stats related to cleanup might be tracked at a system level. I’m thinking along the lines of how often heap_prune_page or the index marking code come across a dead tuple they can’t do anything about yet because it’s still visible. While you could track that per-relation, I’m not sure how helpful that actually is since it’s really a long-running transaction problem.
>
> Similarly, it’d be nice if we had stats about how often all of the auto vac workers were occupied; something that’s also global in nature.
>
>
>
I'll see how these statistics can be calculatedand will add in the patch.
[0]
https://www.postgresql.org/message-id/CAPpHfdvSo3mfH%3D2m4ADCHAuN%3D22SnBY3TrPaPbGKTw3r_Jaw7Q%40mail...
--
Regards,
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] v11-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (19.9K, 3-v11-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 8123aadb292ca2e69e7e779445b61712d9154db7 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Fri, 1 Nov 2024 20:12:03 +0300
Subject: [PATCH 3/3] Machinery for grabbing an extended vacuum statistics on
databases. It transmits vacuum statistical information about each table and
accumulates it for the database which the table belonged.
---
src/backend/catalog/system_views.sql | 26 ++++-
src/backend/utils/activity/pgstat.c | 2 +
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 16 ++++
src/backend/utils/adt/pgstatfuncs.c | 95 +++++++++++++++++++
src/include/catalog/pg_proc.dat | 15 ++-
src/include/pgstat.h | 3 +-
src/test/regress/expected/rules.out | 16 ++++
...ut => vacuum_tables_and_db_statistics.out} | 74 +++++++++++++++
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 64 ++++++++++++-
11 files changed, 306 insertions(+), 8 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (79%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (81%)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index d89b8154e69..d4b99fe658b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1456,4 +1456,28 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_get_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.interrupts AS interrupts
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 79a5c65540c..2a7ad0c930c 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1123,6 +1123,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
if (kind == PGSTAT_KIND_RELATION)
pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ else if (kind == PGSTAT_KIND_DATABASE)
+ pgstat_build_snapshot(PGSTAT_KIND_DATABASE);
}
PG_FINALLY();
{
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 29bc0909748..a060d1a4042 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -430,6 +430,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index f6b072401d4..3c21639cdd4 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -218,6 +218,7 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
if (!pgstat_track_counts)
return;
@@ -231,6 +232,10 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
tabentry->vacuum_ext.interrupts++;
tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.interrupts++;
+ dbentry->vacuum_ext.type = m_type;
}
/*
@@ -244,6 +249,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -297,6 +303,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7fb07a439dc..bf48526e77c 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2341,6 +2341,101 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 13
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",//9
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",//15
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "interrupts",//15
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);//5
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time); //16
+ values[i++] = Float8GetDatum(extvacuum->interrupts);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4ce6ff427c2..d4f0cbecaa8 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12401,7 +12401,7 @@
prosrc => 'gist_stratnum_identity' },
{ oid => '8001',
- descr => 'pg_stat_get_vacuum_tables return stats values',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
@@ -12419,12 +12419,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,interrupts}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 3fb4b3781f4..3e9f4e6fca7 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -174,7 +174,8 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_HEAP = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 8927eb0b074..4970b0b521f 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1881,6 +1881,22 @@ pg_stat_database_conflicts| SELECT oid AS datid,
pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock,
pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot
FROM pg_database d;
+pg_stat_get_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, interrupts);
pg_stat_get_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 79%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index 4295942d84d..870ef163253 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,6 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -212,4 +215,75 @@ SELECT min(relid) FROM pg_stat_get_vacuum_tables(0) where relid > 0;
(1 row)
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 9847a330ed1..1ba32b87cf5 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 81%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index fb3ded309dc..3ebaedb7ed5 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,6 +7,10 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+
-- conditio sine qua non
SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
@@ -143,7 +147,7 @@ FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tabl
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid \gset
-UPDATE vestat SET x = x + 1001;
+UPDATE vestat SET x = x+1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
@@ -157,4 +161,60 @@ FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tabl
SELECT min(relid) FROM pg_stat_get_vacuum_tables(0) where relid > 0;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = current_database();
+
+DROP TABLE vestat CASCADE;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
\ No newline at end of file
--
2.34.1
[text/x-patch] v11-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (42.0K, 4-v11-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 9d4c87145ed1060b39cd17ad8f5bbe0dbec5089a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Fri, 1 Nov 2024 18:24:23 +0300
Subject: [PATCH 2/3] Machinery for grabbing an extended vacuum statistics on
heap and index relations. Remember, statistic on heap and index relations a
bit different (see ExtVacReport to find out more information). The concept of
the ExtVacReport structure has been complicated to store statistic
information for two kinds of relations: for heap and index relations.
ExtVacReportType variable helps to determine what the kind is considering
now.
---
src/backend/access/heap/vacuumlazy.c | 99 +++++++++--
src/backend/catalog/system_views.sql | 32 ++++
src/backend/utils/activity/pgstat.c | 7 +-
src/backend/utils/activity/pgstat_relation.c | 41 +++--
src/backend/utils/adt/pgstatfuncs.c | 126 ++++++++++++-
src/include/catalog/pg_proc.dat | 9 +
src/include/pgstat.h | 52 ++++--
.../vacuum-extending-in-repetable-read.out | 7 +-
src/test/regress/expected/rules.out | 22 +++
.../expected/vacuum_index_statistics.out | 165 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 10 +-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 130 ++++++++++++++
.../regress/sql/vacuum_tables_statistics.sql | 10 +-
14 files changed, 659 insertions(+), 52 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index caa7523484a..370bb14f617 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -166,6 +166,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -244,6 +245,13 @@ typedef struct LVExtStatCounters
PgStat_Counter blocks_hit;
} LVExtStatCounters;
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -394,6 +402,46 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+static void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+static void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -697,14 +745,15 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(rel, &extVacCounters, &extVacReport);
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.pages_frozen = vacrel->set_frozen_pages;
- extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.type = PGSTAT_EXTVAC_HEAP;
+ extVacReport.heap.pages_scanned = vacrel->scanned_pages;
+ extVacReport.heap.pages_removed = vacrel->removed_pages;
+ extVacReport.heap.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.heap.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.heap.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.heap.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.heap.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.heap.index_vacuum_count = vacrel->num_index_scans;
/*
* Report results to the cumulative stats system, too.
@@ -2569,6 +2618,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2587,6 +2640,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2595,6 +2649,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, (void *) vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2619,6 +2680,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2638,12 +2703,20 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3260,7 +3333,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3277,7 +3350,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3293,16 +3366,22 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 292c6629a20..d89b8154e69 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1425,3 +1425,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_get_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 83dade9a5c1..79a5c65540c 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1121,7 +1121,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
PG_TRY();
{
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
- pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ if (kind == PGSTAT_KIND_RELATION)
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
}
PG_FINALLY();
{
@@ -1176,6 +1177,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 9586551656a..f6b072401d4 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -212,7 +212,7 @@ pgstat_drop_relation(Relation rel)
* ---------
*/
void
-pgstat_report_vacuum_error(Oid tableoid)
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -229,6 +229,7 @@ pgstat_report_vacuum_error(Oid tableoid)
tabentry = &shtabentry->stats;
tabentry->vacuum_ext.interrupts++;
+ tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
}
@@ -1039,15 +1040,31 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->pages_frozen += src->pages_frozen;
- dst->pages_all_visible += src->pages_all_visible;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->dead_tuples += src->dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index d2f36081532..7fb07a439dc 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2205,14 +2205,14 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->pages_frozen);
- values[i++] = Int64GetDatum(extvacuum->pages_all_visible);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_frozen);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_all_visible);
+ values[i++] = Int64GetDatum(extvacuum->heap.tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->heap.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->heap.dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->heap.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2231,6 +2231,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",//7
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",//9
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",//15
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);//5
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);//7
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->index.tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time); //16
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 62d17df5770..4ce6ff427c2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12418,4 +12418,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index aaae1ce24cc..3fb4b3781f4 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,11 +169,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_HEAP = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -203,14 +211,38 @@ typedef struct ExtVacReport
/* Interruptions on any errors. */
int32 interrupts;
- int64 pages_scanned; /* number of pages we examined */
- int64 pages_removed; /* number of pages removed by vacuum */
- int64 pages_frozen; /* number of pages marked in VM as frozen */
- int64 pages_all_visible; /* number of pages marked in VM as all-visible */
- int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
- int64 index_vacuum_count; /* number of index vacuumings */
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } heap;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -692,7 +724,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 7cdb79c0ec4..93fe15c01f9 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -9,10 +9,9 @@ step s2_print_vacuum_stats_table:
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
-relname |tuples_deleted|dead_tuples|tuples_frozen
---------------------------+--------------+-----------+-------------
-test_vacuum_stat_isolation| 0| 0| 0
-(1 row)
+relname|tuples_deleted|dead_tuples|tuples_frozen
+-------+--------------+-----------+-------------
+(0 rows)
step s1_begin_repeatable_read:
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 0bd9c6d39b1..8927eb0b074 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1881,6 +1881,28 @@ pg_stat_database_conflicts| SELECT oid AS datid,
pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock,
pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot
FROM pg_database d;
+pg_stat_get_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_get_vacuum_tables| SELECT ns.nspname AS schema,
rel.relname,
stats.relid,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..3a1ae648d0e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,165 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_get_vacuum_indexes(0);
+ min
+-----
+ 0
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index a3d8a4301f3..4295942d84d 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -23,8 +23,6 @@ SELECT pg_stat_force_next_flush();
(1 row)
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -156,6 +154,14 @@ WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
vestat | t | t | f | t | t
(1 row)
+DROP TABLE vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
-- must be empty
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 977a0472027..9847a330ed1 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..e3cddee6601
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,130 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+SELECT min(relid) FROM pg_stat_get_vacuum_indexes(0);
+
+DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
index 88208ea82cd..fb3ded309dc 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -17,8 +17,7 @@ SET track_functions TO 'all';
SELECT pg_stat_force_next_flush();
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
+
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -126,13 +125,16 @@ SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS
FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+DROP TABLE vestat;
+SELECT pg_stat_force_next_flush();
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
-- must be empty
SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
-
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- backend defreezed pages
@@ -143,14 +145,12 @@ FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tabl
UPDATE vestat SET x = x + 1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_get_vacuum_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' \gset
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-
-- vacuum freezed pages
SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
--
2.34.1
[text/x-patch] v11-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (67.7K, 5-v11-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 257054d073cc48c728a7f59cf85393e82fe6f3d7 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Fri, 1 Nov 2024 13:39:58 +0300
Subject: [PATCH 1/3] Machinery for grabbing an extended vacuum statistics on
heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 147 +++++++++++-
src/backend/access/heap/visibilitymap.c | 13 ++
src/backend/catalog/system_views.sql | 48 +++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 32 ++-
src/backend/utils/activity/pgstat_relation.c | 70 +++++-
src/backend/utils/adt/pgstatfuncs.c | 171 ++++++++++++++
src/backend/utils/error/elog.c | 13 ++
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 79 ++++++-
src/include/utils/elog.h | 1 +
src/include/utils/pgstat_internal.h | 2 +-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 51 +++++
src/test/regress/expected/rules.out | 40 +++-
.../expected/vacuum_tables_statistics.out | 209 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 160 ++++++++++++++
21 files changed, 1102 insertions(+), 17 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 793bd33cb4d..caa7523484a 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -165,6 +165,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -192,6 +193,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -224,6 +227,22 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Cut-off values of parameters which changes implicitly during a vacuum
+ * process.
+ * Vacuum can't control their values, so we should store them before and after
+ * the processing.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz time;
+ PGRUsage ru;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -277,6 +296,103 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+ PGRUsage ru0;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ pg_rusage_init(&ru0);
+ starttime = GetCurrentTimestamp();
+
+ counters->ru = ru0;
+ counters->time = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->time, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -309,6 +425,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -327,7 +445,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -344,6 +462,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -411,6 +530,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -572,6 +693,19 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -586,7 +720,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1378,6 +1513,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2275,11 +2412,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -3120,6 +3259,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3135,6 +3276,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33c..d72cade60a4 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,18 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Initially, it didn't matter what type of flags (all-visible or frozen) we received,
+ * we just performed a reverse concatenation operation. But this information is very important
+ * for vacuum statistics. We need to find out this usingthe bit concatenation operation
+ * with the VISIBILITYMAP_ALL_VISIBLE and VISIBILITYMAP_ALL_FROZEN masks,
+ * and where the desired one matches, we increment the value there.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 3456b821bc5..292c6629a20 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -691,7 +691,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_vacuum_count(C.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(C.oid) AS autovacuum_count,
pg_stat_get_analyze_count(C.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1379,3 +1381,47 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_get_vacuum_tables AS
+SELECT
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.pages_frozen AS pages_frozen,
+ stats.pages_all_visible AS pages_all_visible,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.dead_tuples AS dead_tuples,
+
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 86f36b36954..1cff446fb92 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -102,6 +102,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2418,6 +2421,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 4fd6574e129..7f7c7c16e23 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1048,6 +1048,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index be48432cc38..83dade9a5c1 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -189,7 +189,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -259,7 +259,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -878,7 +877,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -944,7 +942,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1060,7 +1058,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1110,8 +1108,30 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 36d3adf7310..9586551656a 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -203,12 +205,40 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.interrupts++;
+ pgstat_unlock_entry(entry_ref);
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -232,6 +262,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -860,6 +892,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -983,3 +1018,36 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+ dst->interrupts += src->interrupts;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->dead_tuples += src->dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index f7b50e0b5af..d2f36081532 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -31,6 +31,42 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+#include "utils/pgstat_internal.h"
+
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -106,6 +142,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_TIMESTAMPTZ(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2063,3 +2105,132 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 22
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_all_visible",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->pages_frozen);
+ values[i++] = Int64GetDatum(extvacuum->pages_all_visible);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 8acca3e0a0b..fe554547f5b 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 1ec0d6f6b5f..62d17df5770 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12400,4 +12400,22 @@
proargtypes => 'int2',
prosrc => 'gist_stratnum_identity' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables return stats values',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index df53fa2d4f9..aaae1ce24cc 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,6 +169,50 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ int64 total_blks_read; /* number of pages that were missed in shared buffers during a vacuum of specific relation */
+ int64 total_blks_hit; /* number of pages that were found in shared buffers during a vacuum of specific relation */
+ int64 total_blks_dirtied; /* number of pages marked as 'Dirty' during a vacuum of specific relation. */
+ int64 total_blks_written; /* number of pages written during a vacuum of specific relation. */
+
+ int64 blks_fetched; /* number of a relation blocks, fetched during the vacuum. */
+ int64 blks_hit; /* number of a relation blocks, found in shared buffers during the vacuum. */
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 interrupts;
+
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -209,6 +253,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -267,7 +321,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB1
typedef struct PgStat_ArchiverStats
{
@@ -388,6 +442,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -461,6 +517,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -626,10 +687,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -677,6 +740,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -694,7 +768,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
/*
* Functions in pgstat_replslot.c
*/
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index e54eca5b489..e752c0ce015 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 61b2e1f96b2..2c0e55d63f3 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -573,7 +573,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
*/
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..7cdb79c0ec4
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 100| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 100| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..e93dd4f626c 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5facb2c862c
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,51 @@
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
+# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2b47013f113..0bd9c6d39b1 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1804,7 +1804,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_vacuum_count(c.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(c.oid) AS autovacuum_count,
pg_stat_get_analyze_count(c.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -1879,6 +1881,34 @@ pg_stat_database_conflicts| SELECT oid AS datid,
pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock,
pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot
FROM pg_database d;
+pg_stat_get_vacuum_tables| SELECT ns.nspname AS schema,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_gssapi| SELECT pid,
gss_auth AS gss_authenticated,
gss_princ AS principal,
@@ -2186,7 +2216,9 @@ pg_stat_sys_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2234,7 +2266,9 @@ pg_stat_user_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..a3d8a4301f3
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,209 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | t | t
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | f | f
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_get_vacuum_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' \gset
+ERROR: column "rev_all_frozen_pages" does not exist
+LINE 1: ...LECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_fr...
+ ^
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ t | t | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_get_vacuum_tables(0) where relid > 0;
+ min
+-----
+
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 81e4222d26a..977a0472027 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..88208ea82cd
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,160 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_get_vacuum_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+SELECT min(relid) FROM pg_stat_get_vacuum_tables(0) where relid > 0;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v11-0004-Add-documentation-about-the-system-views-that-are-us.patch (27.1K, 6-v11-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 4134c047c63f2b6acc3fe119f7fbd3d66c085bea Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:47:55 +0300
Subject: [PATCH] Add documentation about the system views that are used in the
machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 747 ++++++++++++++++++
.../vacuum-extending-in-repetable-read.out | 6 +-
.../vacuum-extending-in-repetable-read.spec | 4 +-
3 files changed, 752 insertions(+), 5 deletions(-)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 61d28e701f2..b74c53bb000 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5064,4 +5064,751 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-get-vacuum-database">
+ <title><structname>pg_stat_get_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-database">
+ <primary>pg_stat_get_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-get-vacuum-indexes">
+ <title><structname>pg_stat_get_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-indexes">
+ <primary>pg_stat_get_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this index
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-get-vacuum-tables">
+ <title><structname>pg_stat_get_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-tables">
+ <primary>pg_stat_get_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-frozen in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_all_visible</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-visible in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this table
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
+
</chapter>
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 93fe15c01f9..a7794023508 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -6,7 +6,7 @@ step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, iv
step s2_print_vacuum_stats_table:
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
relname|tuples_deleted|dead_tuples|tuples_frozen
@@ -28,7 +28,7 @@ step s2_vacuum: VACUUM test_vacuum_stat_isolation;
step s2_print_vacuum_stats_table:
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
relname |tuples_deleted|dead_tuples|tuples_frozen
@@ -42,7 +42,7 @@ step s2_vacuum: VACUUM test_vacuum_stat_isolation;
step s2_print_vacuum_stats_table:
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
relname |tuples_deleted|dead_tuples|tuples_frozen
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5facb2c862c..6e33df46480 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -1,4 +1,4 @@
-# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_get_vacuum_tables.
# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
# by the value of the cleared tuples that the vacuum managed to clear.
@@ -33,7 +33,7 @@ step s2_print_vacuum_stats_table
{
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
}
--
2.34.1
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-11-08 19:34 Jim Nasby <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Jim Nasby @ 2024-11-08 19:34 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Andrei Zubkov <[email protected]>; Alexander Korotkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
> On Nov 2, 2024, at 7:22 AM, Alena Rybakina <[email protected]> wrote:
>
>>> The second is the interrupts field. It is needed for monitoring to know
>>> do we have them or not, so tracking them on the database level will do
>>> the trick. Interrupt is quite rare event, so once the monitoring system
>>> will catch one the DBA can go to the server log for the details.
>> Just to confirm… by “interrupt” you mean vacuum encountered an error?
> Yes it is.
In that case I feel rather strongly that we should label that as “errors”. “Interrupt” could mean a few different things, but “error” is very clear.
> I updated patches. I excluded system and user time statistics and save number of interrupts only for database.
> I removed the ability to get statistics for all tables, now they can only be obtained for an oid table [0], as suggested here. I also renamed the statistics from pg_stat_vacuum_tables to pg_stat_get_vacuum_tables and similarly for indexes and databases. I noticed that that’s what they’re mostly called. Ready for discussion.
>
I think it’s better that the views follow the existing naming conventions (which don’t include “_get_”; only the functions have that in their names). Assuming that, the only question becomes pg_stat_vacuum_* vs pg_stat_*_vacuum. Given the existing precedent of pg_statio_*, I’m inclined to go with pg_stat_vacuum_*.
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-11-10 20:09 Alena Rybakina <[email protected]>
parent: Jim Nasby <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2024-11-10 20:09 UTC (permalink / raw)
To: Jim Nasby <[email protected]>; +Cc: Andrei Zubkov <[email protected]>; Alexander Korotkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
On 08.11.2024 22:34, Jim Nasby wrote:
>
>> On Nov 2, 2024, at 7:22 AM, Alena Rybakina
>> <[email protected]> wrote:
>>
>>>> The second is the interrupts field. It is needed for monitoring to know
>>>> do we have them or not, so tracking them on the database level will do
>>>> the trick. Interrupt is quite rare event, so once the monitoring system
>>>> will catch one the DBA can go to the server log for the details.
>>> Just to confirm… by “interrupt” you mean vacuum encountered an error?
>> Yes it is.
> In that case I feel rather strongly that we should label that as
> “errors”. “Interrupt” could mean a few different things, but “error”
> is very clear.
>>
>> I updated patches. I excluded system and user time statistics and
>> save number of interrupts only for database.I removed the ability to
>> get statistics for all tables, now they can only be obtained for an
>> oid table [0], as suggested here. I also renamed the statistics from
>> pg_stat_vacuum_tables to pg_stat_get_vacuum_tables and similarly for
>> indexes and databases. I noticed that that’s what they’re mostly
>> called. Ready for discussion.
>>
> I think it’s better that the views follow the existing naming
> conventions (which don’t include “_get_”; only the functions have that
> in their names). Assuming that, the only question becomes
> pg_stat_vacuum_* vs pg_stat_*_vacuum. Given the existing precedent of
> pg_statio_*, I’m inclined to go with pg_stat_vacuum_*.
I have fixed it.
--
Regards,
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] v12-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (19.8K, 3-v12-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 1b4ac690ea9fbe3180436cb4d0292eeb31669400 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 10 Nov 2024 23:04:36 +0300
Subject: [PATCH 3/3] Machinery for grabbing an extended vacuum statistics on
databases. It transmits vacuum statistical information about each table and
accumulates it for the database which the table belonged.
---
src/backend/catalog/system_views.sql | 26 ++++-
src/backend/utils/activity/pgstat.c | 2 +
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 16 ++++
src/backend/utils/adt/pgstatfuncs.c | 95 +++++++++++++++++++
src/include/catalog/pg_proc.dat | 15 ++-
src/include/pgstat.h | 3 +-
src/test/regress/expected/rules.out | 16 ++++
...ut => vacuum_tables_and_db_statistics.out} | 74 +++++++++++++++
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 64 ++++++++++++-
11 files changed, 306 insertions(+), 8 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (79%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (81%)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index b9a87e7ca58..f1a1e90ba90 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1456,4 +1456,28 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_get_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.errors AS errors
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 551dcfa3198..5b81fbba12a 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1126,6 +1126,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
if (kind == PGSTAT_KIND_RELATION)
pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ else if (kind == PGSTAT_KIND_DATABASE)
+ pgstat_build_snapshot(PGSTAT_KIND_DATABASE);
}
PG_FINALLY();
{
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 29bc0909748..a060d1a4042 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -430,6 +430,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index efd936c663a..5187307b848 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -218,6 +218,7 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
if (!pgstat_track_counts)
return;
@@ -231,6 +232,10 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
tabentry->vacuum_ext.interrupts++;
tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.errors++;
+ dbentry->vacuum_ext.type = m_type;
}
/*
@@ -244,6 +249,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -297,6 +303,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index eddee8972cc..d2affd22f46 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2341,6 +2341,101 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 13
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f27f9981773..b5b7baede31 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12393,7 +12393,7 @@
prosrc => 'gist_stratnum_identity' },
{ oid => '8001',
- descr => 'pg_stat_get_vacuum_tables return stats values',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
@@ -12411,12 +12411,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 96379d8bcc3..d8875ea4177 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -174,7 +174,8 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_HEAP = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 8927eb0b074..4970b0b521f 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1881,6 +1881,22 @@ pg_stat_database_conflicts| SELECT oid AS datid,
pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock,
pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot
FROM pg_database d;
+pg_stat_get_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, interrupts);
pg_stat_get_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 79%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index 4295942d84d..870ef163253 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,6 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -212,4 +215,75 @@ SELECT min(relid) FROM pg_stat_get_vacuum_tables(0) where relid > 0;
(1 row)
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 9847a330ed1..1ba32b87cf5 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 81%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index fb3ded309dc..3ebaedb7ed5 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,6 +7,10 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+
-- conditio sine qua non
SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
@@ -143,7 +147,7 @@ FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tabl
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid \gset
-UPDATE vestat SET x = x + 1001;
+UPDATE vestat SET x = x+1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
@@ -157,4 +161,60 @@ FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tabl
SELECT min(relid) FROM pg_stat_get_vacuum_tables(0) where relid > 0;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = current_database();
+
+DROP TABLE vestat CASCADE;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
\ No newline at end of file
--
2.34.1
[text/x-patch] v12-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (42.0K, 4-v12-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From e9a84f17afddbe478bbd2d81af7a80779e5ecf65 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 10 Nov 2024 23:00:50 +0300
Subject: [PATCH 2/3] Machinery for grabbing an extended vacuum statistics on
heap and index relations. Remember, statistic on heap and index relations a
bit different (see ExtVacReport to find out more information). The concept of
the ExtVacReport structure has been complicated to store statistic
information for two kinds of relations: for heap and index relations.
ExtVacReportType variable helps to determine what the kind is considering
now.
---
src/backend/access/heap/vacuumlazy.c | 99 +++++++++--
src/backend/catalog/system_views.sql | 32 ++++
src/backend/utils/activity/pgstat.c | 7 +-
src/backend/utils/activity/pgstat_relation.c | 41 +++--
src/backend/utils/adt/pgstatfuncs.c | 126 ++++++++++++-
src/include/catalog/pg_proc.dat | 9 +
src/include/pgstat.h | 52 ++++--
.../vacuum-extending-in-repetable-read.out | 7 +-
src/test/regress/expected/rules.out | 22 +++
.../expected/vacuum_index_statistics.out | 165 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 10 +-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 130 ++++++++++++++
.../regress/sql/vacuum_tables_statistics.sql | 10 +-
14 files changed, 659 insertions(+), 52 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index caa7523484a..370bb14f617 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -166,6 +166,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -244,6 +245,13 @@ typedef struct LVExtStatCounters
PgStat_Counter blocks_hit;
} LVExtStatCounters;
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -394,6 +402,46 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+static void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+static void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -697,14 +745,15 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(rel, &extVacCounters, &extVacReport);
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.pages_frozen = vacrel->set_frozen_pages;
- extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.type = PGSTAT_EXTVAC_HEAP;
+ extVacReport.heap.pages_scanned = vacrel->scanned_pages;
+ extVacReport.heap.pages_removed = vacrel->removed_pages;
+ extVacReport.heap.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.heap.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.heap.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.heap.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.heap.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.heap.index_vacuum_count = vacrel->num_index_scans;
/*
* Report results to the cumulative stats system, too.
@@ -2569,6 +2618,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2587,6 +2640,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2595,6 +2649,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, (void *) vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2619,6 +2680,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2638,12 +2703,20 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3260,7 +3333,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3277,7 +3350,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3293,16 +3366,22 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 0ac210d03f7..b9a87e7ca58 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1425,3 +1425,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_get_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 3f2e9f54685..551dcfa3198 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1124,7 +1124,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
PG_TRY();
{
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
- pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ if (kind == PGSTAT_KIND_RELATION)
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
}
PG_FINALLY();
{
@@ -1179,6 +1180,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 2a2650ef2ca..efd936c663a 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -212,7 +212,7 @@ pgstat_drop_relation(Relation rel)
* ---------
*/
void
-pgstat_report_vacuum_error(Oid tableoid)
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -229,6 +229,7 @@ pgstat_report_vacuum_error(Oid tableoid)
tabentry = &shtabentry->stats;
tabentry->vacuum_ext.interrupts++;
+ tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
}
@@ -1036,15 +1037,31 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->pages_frozen += src->pages_frozen;
- dst->pages_all_visible += src->pages_all_visible;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->dead_tuples += src->dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index d2f36081532..eddee8972cc 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2205,14 +2205,14 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->pages_frozen);
- values[i++] = Int64GetDatum(extvacuum->pages_all_visible);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_frozen);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_all_visible);
+ values[i++] = Int64GetDatum(extvacuum->heap.tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->heap.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->heap.dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->heap.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2231,6 +2231,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->index.tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f68072d32b9..f27f9981773 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12410,4 +12410,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index e8350ca438c..96379d8bcc3 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,11 +169,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_HEAP = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -203,14 +211,38 @@ typedef struct ExtVacReport
/* Interruptions on any errors. */
int32 errors;
- int64 pages_scanned; /* number of pages we examined */
- int64 pages_removed; /* number of pages removed by vacuum */
- int64 pages_frozen; /* number of pages marked in VM as frozen */
- int64 pages_all_visible; /* number of pages marked in VM as all-visible */
- int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
- int64 index_vacuum_count; /* number of index vacuumings */
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } heap;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -692,7 +724,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 7cdb79c0ec4..93fe15c01f9 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -9,10 +9,9 @@ step s2_print_vacuum_stats_table:
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
-relname |tuples_deleted|dead_tuples|tuples_frozen
---------------------------+--------------+-----------+-------------
-test_vacuum_stat_isolation| 0| 0| 0
-(1 row)
+relname|tuples_deleted|dead_tuples|tuples_frozen
+-------+--------------+-----------+-------------
+(0 rows)
step s1_begin_repeatable_read:
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 0bd9c6d39b1..8927eb0b074 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1881,6 +1881,28 @@ pg_stat_database_conflicts| SELECT oid AS datid,
pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock,
pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot
FROM pg_database d;
+pg_stat_get_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_get_vacuum_tables| SELECT ns.nspname AS schema,
rel.relname,
stats.relid,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..3a1ae648d0e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,165 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_get_vacuum_indexes(0);
+ min
+-----
+ 0
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index a3d8a4301f3..4295942d84d 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -23,8 +23,6 @@ SELECT pg_stat_force_next_flush();
(1 row)
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -156,6 +154,14 @@ WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
vestat | t | t | f | t | t
(1 row)
+DROP TABLE vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
-- must be empty
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 977a0472027..9847a330ed1 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..e3cddee6601
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,130 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+SELECT min(relid) FROM pg_stat_get_vacuum_indexes(0);
+
+DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
index 88208ea82cd..fb3ded309dc 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -17,8 +17,7 @@ SET track_functions TO 'all';
SELECT pg_stat_force_next_flush();
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
+
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -126,13 +125,16 @@ SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS
FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+DROP TABLE vestat;
+SELECT pg_stat_force_next_flush();
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
-- must be empty
SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
-
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- backend defreezed pages
@@ -143,14 +145,12 @@ FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tabl
UPDATE vestat SET x = x + 1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_get_vacuum_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' \gset
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-
-- vacuum freezed pages
SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
--
2.34.1
[text/x-patch] v12-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (67.7K, 5-v12-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 40dcc17de42592ae4bd83deaf40574b3210db0f5 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 10 Nov 2024 22:58:08 +0300
Subject: [PATCH 1/3] Machinery for grabbing an extended vacuum statistics on
heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 147 +++++++++++-
src/backend/access/heap/visibilitymap.c | 13 ++
src/backend/catalog/system_views.sql | 48 +++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 32 ++-
src/backend/utils/activity/pgstat_relation.c | 70 +++++-
src/backend/utils/adt/pgstatfuncs.c | 171 ++++++++++++++
src/backend/utils/error/elog.c | 13 ++
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 79 ++++++-
src/include/utils/elog.h | 1 +
src/include/utils/pgstat_internal.h | 2 +-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 51 +++++
src/test/regress/expected/rules.out | 40 +++-
.../expected/vacuum_tables_statistics.out | 209 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 160 ++++++++++++++
21 files changed, 1102 insertions(+), 17 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 793bd33cb4d..caa7523484a 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -165,6 +165,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -192,6 +193,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -224,6 +227,22 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Cut-off values of parameters which changes implicitly during a vacuum
+ * process.
+ * Vacuum can't control their values, so we should store them before and after
+ * the processing.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz time;
+ PGRUsage ru;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -277,6 +296,103 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+ PGRUsage ru0;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ pg_rusage_init(&ru0);
+ starttime = GetCurrentTimestamp();
+
+ counters->ru = ru0;
+ counters->time = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->time, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -309,6 +425,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -327,7 +445,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -344,6 +462,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -411,6 +530,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -572,6 +693,19 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -586,7 +720,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1378,6 +1513,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2275,11 +2412,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -3120,6 +3259,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3135,6 +3276,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33c..d72cade60a4 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,18 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Initially, it didn't matter what type of flags (all-visible or frozen) we received,
+ * we just performed a reverse concatenation operation. But this information is very important
+ * for vacuum statistics. We need to find out this usingthe bit concatenation operation
+ * with the VISIBILITYMAP_ALL_VISIBLE and VISIBILITYMAP_ALL_FROZEN masks,
+ * and where the desired one matches, we increment the value there.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 3456b821bc5..0ac210d03f7 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -691,7 +691,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_vacuum_count(C.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(C.oid) AS autovacuum_count,
pg_stat_get_analyze_count(C.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1379,3 +1381,47 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.pages_frozen AS pages_frozen,
+ stats.pages_all_visible AS pages_all_visible,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.dead_tuples AS dead_tuples,
+
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 86f36b36954..1cff446fb92 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -102,6 +102,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2418,6 +2421,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 4fd6574e129..7f7c7c16e23 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1048,6 +1048,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index ea8c5691e87..3f2e9f54685 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -189,7 +189,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -259,7 +259,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -878,7 +877,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -947,7 +945,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1063,7 +1061,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1113,8 +1111,30 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index faba8b64d23..2a2650ef2ca 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -203,12 +205,40 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.interrupts++;
+ pgstat_unlock_entry(entry_ref);
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -232,6 +262,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -857,6 +889,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -980,3 +1015,36 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+ dst->errors += src->errors;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->dead_tuples += src->dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index f7b50e0b5af..d2f36081532 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -31,6 +31,42 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+#include "utils/pgstat_internal.h"
+
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -106,6 +142,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_TIMESTAMPTZ(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2063,3 +2105,132 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 22
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_all_visible",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->pages_frozen);
+ values[i++] = Int64GetDatum(extvacuum->pages_all_visible);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 8acca3e0a0b..fe554547f5b 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f23321a41f1..f68072d32b9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12392,4 +12392,22 @@
proargtypes => 'int2',
prosrc => 'gist_stratnum_identity' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables return stats values',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index df53fa2d4f9..e8350ca438c 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,6 +169,50 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ int64 total_blks_read; /* number of pages that were missed in shared buffers during a vacuum of specific relation */
+ int64 total_blks_hit; /* number of pages that were found in shared buffers during a vacuum of specific relation */
+ int64 total_blks_dirtied; /* number of pages marked as 'Dirty' during a vacuum of specific relation. */
+ int64 total_blks_written; /* number of pages written during a vacuum of specific relation. */
+
+ int64 blks_fetched; /* number of a relation blocks, fetched during the vacuum. */
+ int64 blks_hit; /* number of a relation blocks, found in shared buffers during the vacuum. */
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 errors;
+
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -209,6 +253,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -267,7 +321,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB1
typedef struct PgStat_ArchiverStats
{
@@ -388,6 +442,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -461,6 +517,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -626,10 +687,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -677,6 +740,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -694,7 +768,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
/*
* Functions in pgstat_replslot.c
*/
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index e54eca5b489..e752c0ce015 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 61b2e1f96b2..2c0e55d63f3 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -573,7 +573,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
*/
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..7cdb79c0ec4
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 100| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 100| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..e93dd4f626c 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5facb2c862c
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,51 @@
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
+# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2b47013f113..0bd9c6d39b1 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1804,7 +1804,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_vacuum_count(c.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(c.oid) AS autovacuum_count,
pg_stat_get_analyze_count(c.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -1879,6 +1881,34 @@ pg_stat_database_conflicts| SELECT oid AS datid,
pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock,
pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot
FROM pg_database d;
+pg_stat_get_vacuum_tables| SELECT ns.nspname AS schema,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_gssapi| SELECT pid,
gss_auth AS gss_authenticated,
gss_princ AS principal,
@@ -2186,7 +2216,9 @@ pg_stat_sys_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2234,7 +2266,9 @@ pg_stat_user_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..a3d8a4301f3
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,209 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | t | t
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | f | f
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_get_vacuum_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' \gset
+ERROR: column "rev_all_frozen_pages" does not exist
+LINE 1: ...LECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_fr...
+ ^
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ t | t | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_get_vacuum_tables(0) where relid > 0;
+ min
+-----
+
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 81e4222d26a..977a0472027 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..88208ea82cd
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,160 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_get_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_get_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_get_vacuum_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_get_vacuum_tables, pg_stat_all_tables WHERE pg_stat_get_vacuum_tables.relname = 'vestat' and pg_stat_get_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+SELECT min(relid) FROM pg_stat_get_vacuum_tables(0) where relid > 0;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v12-0004-Add-documentation-about-the-system-views-that-are-us.patch (27.1K, 6-v12-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 4134c047c63f2b6acc3fe119f7fbd3d66c085bea Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:47:55 +0300
Subject: [PATCH] Add documentation about the system views that are used in the
machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 747 ++++++++++++++++++
.../vacuum-extending-in-repetable-read.out | 6 +-
.../vacuum-extending-in-repetable-read.spec | 4 +-
3 files changed, 752 insertions(+), 5 deletions(-)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 61d28e701f2..b74c53bb000 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5064,4 +5064,751 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-get-vacuum-database">
+ <title><structname>pg_stat_get_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-database">
+ <primary>pg_stat_get_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-get-vacuum-indexes">
+ <title><structname>pg_stat_get_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-indexes">
+ <primary>pg_stat_get_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this index
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-get-vacuum-tables">
+ <title><structname>pg_stat_get_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-tables">
+ <primary>pg_stat_get_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-frozen in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_all_visible</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-visible in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this table
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
+
</chapter>
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 93fe15c01f9..a7794023508 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -6,7 +6,7 @@ step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, iv
step s2_print_vacuum_stats_table:
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
relname|tuples_deleted|dead_tuples|tuples_frozen
@@ -28,7 +28,7 @@ step s2_vacuum: VACUUM test_vacuum_stat_isolation;
step s2_print_vacuum_stats_table:
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
relname |tuples_deleted|dead_tuples|tuples_frozen
@@ -42,7 +42,7 @@ step s2_vacuum: VACUUM test_vacuum_stat_isolation;
step s2_print_vacuum_stats_table:
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
relname |tuples_deleted|dead_tuples|tuples_frozen
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5facb2c862c..6e33df46480 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -1,4 +1,4 @@
-# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_get_vacuum_tables.
# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
# by the value of the cleared tuples that the vacuum managed to clear.
@@ -33,7 +33,7 @@ step s2_print_vacuum_stats_table
{
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
}
--
2.34.1
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-11-13 00:24 Jim Nasby <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Jim Nasby @ 2024-11-13 00:24 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Andrei Zubkov <[email protected]>; Alexander Korotkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
On Nov 10, 2024, at 2:09 PM, Alena Rybakina <[email protected]> wrote:
>
> On 08.11.2024 22:34, Jim Nasby wrote:
>>
>>> On Nov 2, 2024, at 7:22 AM, Alena Rybakina <[email protected]> <mailto:[email protected]> wrote:
>>>
>>>>> The second is the interrupts field. It is needed for monitoring to know
>>>>> do we have them or not, so tracking them on the database level will do
>>>>> the trick. Interrupt is quite rare event, so once the monitoring system
>>>>> will catch one the DBA can go to the server log for the details.
>>>> Just to confirm… by “interrupt” you mean vacuum encountered an error?
>>> Yes it is.
>> In that case I feel rather strongly that we should label that as “errors”. “Interrupt” could mean a few different things, but “error” is very clear.
>>> I updated patches. I excluded system and user time statistics and save number of interrupts only for database.
>>> I removed the ability to get statistics for all tables, now they can only be obtained for an oid table [0], as suggested here. I also renamed the statistics from pg_stat_vacuum_tables to pg_stat_get_vacuum_tables and similarly for indexes and databases. I noticed that that’s what they’re mostly called. Ready for discussion.
>>>
>> I think it’s better that the views follow the existing naming conventions (which don’t include “_get_”; only the functions have that in their names). Assuming that, the only question becomes pg_stat_vacuum_* vs pg_stat_*_vacuum. Given the existing precedent of pg_statio_*, I’m inclined to go with pg_stat_vacuum_*.
> I have fixed it.
I’ve reviewed and made some cosmetic changes to patch 1, though of note it looks like an effort has been made to keep stat_reset_timestamp at the end of PgStat_StatDBEntry, so I re-arranged that. I also removed some obviously dead code. It appears that pgstat_update_snapshot(), InitSnapshotIterator() and ScanStatSnapshot() are also dead, but I’ve left it in incase I’m missing something. The tests are also failing for me because a number of psql variables aren’t set.
I do think we should separate out the counts for deleted but still visible tuples vs tuples where we couldn’t get a cleanup lock (in other words, recently_dead_tuples and missed_dead_tuples from LVRelState). I realize that’s a departure from how some of the existing reporting works, but IMO combining them together isn’t a pattern we should be repeating since they mean completely different things. Towards that end I did remove missed_dead_tuples from the reporting, and renamed ExtVacReport.dead_tuples to recently_dead_tuples, but I stopped short of creating a separate entry for missed_dead_tuples. Note that while recently_dead_tuples is really a global thing (so only needs to be reported at a global (or at most per-database) level, but missed_dead_tuples should really be at a per-table level.
Updated 0001-v13 attached, as well as the diff between v12 and v13.

Attachments:
[application/octet-stream] vacuum_stats_0001_v12_v13.patch (9.3K, 3-vacuum_stats_0001_v12_v13.patch)
download | inline diff:
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index caa7523484..85e9b26040 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -228,15 +228,11 @@ typedef struct LVSavedErrInfo
} LVSavedErrInfo;
/*
- * Cut-off values of parameters which changes implicitly during a vacuum
- * process.
- * Vacuum can't control their values, so we should store them before and after
- * the processing.
+ * Counters and usage data for extended stats tracking.
*/
typedef struct LVExtStatCounters
{
- TimestampTz time;
- PGRUsage ru;
+ TimestampTz starttime;
WalUsage walusage;
BufferUsage bufusage;
double VacuumDelayTime;
@@ -296,26 +292,21 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
-/* ----------
+/*
* extvac_stats_start() -
*
- * Save cut-off values of extended vacuum counters before start of a relation
- * processing.
- * ----------
+ * Save extended stats counters before start of relation processing.
*/
static void
extvac_stats_start(Relation rel, LVExtStatCounters *counters)
{
TimestampTz starttime;
- PGRUsage ru0;
memset(counters, 0, sizeof(LVExtStatCounters));
- pg_rusage_init(&ru0);
starttime = GetCurrentTimestamp();
- counters->ru = ru0;
- counters->time = starttime;
+ counters->starttime = starttime;
counters->walusage = pgWalUsage;
counters->bufusage = pgBufferUsage;
counters->VacuumDelayTime = VacuumDelayTime;
@@ -324,7 +315,7 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
if (!rel->pgstat_info || !pgstat_track_counts)
/*
- * if something goes wrong or an user doesn't want to track a database
+ * if something goes wrong or user doesn't want to track a database
* activity - just suppress it.
*/
return;
@@ -333,11 +324,10 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
}
-/* ----------
+/*
* extvac_stats_end() -
*
- * Called to finish an extended vacuum statistic gathering and form a report.
- * ----------
+ * Called to finish an extended vacuum statistic gathering and form a report.
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters *counters,
@@ -357,7 +347,7 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
endtime = GetCurrentTimestamp();
- TimestampDifference(counters->time, endtime, &secs, &usecs);
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
memset(report, 0, sizeof(ExtVacReport));
@@ -703,7 +693,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
extVacReport.tuples_deleted = vacrel->tuples_deleted;
extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
extVacReport.index_vacuum_count = vacrel->num_index_scans;
/*
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index d72cade60a..d25c9a3679 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -162,12 +162,9 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
/*
- * Initially, it didn't matter what type of flags (all-visible or frozen) we received,
- * we just performed a reverse concatenation operation. But this information is very important
- * for vacuum statistics. We need to find out this usingthe bit concatenation operation
- * with the VISIBILITYMAP_ALL_VISIBLE and VISIBILITYMAP_ALL_FROZEN masks,
- * and where the desired one matches, we increment the value there.
- */
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
pgstat_count_vm_rev_all_visible(rel);
if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 2a2650ef2c..5dd8275671 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -205,11 +205,8 @@ pgstat_drop_relation(Relation rel)
}
}
-/* ---------
- * pgstat_report_vacuum_error() -
- *
- * Tell the collector about an (auto)vacuum interruption.
- * ---------
+/*
+ * Report an error while vacuuming.
*/
void
pgstat_report_vacuum_error(Oid tableoid)
@@ -228,7 +225,7 @@ pgstat_report_vacuum_error(Oid tableoid)
shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
tabentry = &shtabentry->stats;
- tabentry->vacuum_ext.interrupts++;
+ tabentry->vacuum_ext.errors++;
pgstat_unlock_entry(entry_ref);
}
@@ -1045,6 +1042,6 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
dst->pages_all_visible += src->pages_all_visible;
dst->tuples_deleted += src->tuples_deleted;
dst->tuples_frozen += src->tuples_frozen;
- dst->dead_tuples += src->dead_tuples;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
dst->index_vacuum_count += src->index_vacuum_count;
-}
\ No newline at end of file
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index e139ae5134..ab3fba3314 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2217,7 +2217,7 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
values[i++] = Int64GetDatum(extvacuum->pages_all_visible);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
@@ -2239,4 +2239,4 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
-}
\ No newline at end of file
+}
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 5d1a3c536d..a6f363ac0d 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,25 +169,22 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
-/* ----------
- *
+/*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
- * pages_removed is the amount by which the physically shrank,
- * if any (ie the change in its total size on disk)
- * pages_deleted refer to free space within the index file
- * ----------
+ * Additional statistics of vacuum processing over a single heap relation.
*/
typedef struct ExtVacReport
{
- int64 total_blks_read; /* number of pages that were missed in shared buffers during a vacuum of specific relation */
- int64 total_blks_hit; /* number of pages that were found in shared buffers during a vacuum of specific relation */
- int64 total_blks_dirtied; /* number of pages marked as 'Dirty' during a vacuum of specific relation. */
- int64 total_blks_written; /* number of pages written during a vacuum of specific relation. */
+ /* number of blocks missed, hit, dirtied and written */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
- int64 blks_fetched; /* number of a relation blocks, fetched during the vacuum. */
- int64 blks_hit; /* number of a relation blocks, found in shared buffers during the vacuum. */
+ /* blocks missed and hit for just the heap */
+ int64 blks_fetched;
+ int64 blks_hit;
/* Vacuum WAL usage stats */
int64 wal_records; /* wal usage: number of WAL records */
@@ -203,13 +200,13 @@ typedef struct ExtVacReport
/* Interruptions on any errors. */
int32 errors;
- int64 pages_scanned; /* number of pages we examined */
- int64 pages_removed; /* number of pages removed by vacuum */
- int64 pages_frozen; /* number of pages marked in VM as frozen */
- int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
int64 tuples_deleted; /* tuples deleted by vacuum */
int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
int64 index_vacuum_count; /* number of index vacuumings */
} ExtVacReport;
@@ -442,10 +439,9 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
PgStat_Counter parallel_workers_to_launch;
PgStat_Counter parallel_workers_launched;
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
TimestampTz stat_reset_timestamp;
-
- ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
[application/octet-stream] vacuum_stats_0001_v13.patch (38.3K, 5-vacuum_stats_0001_v13.patch)
download | inline diff:
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 793bd33cb4..85e9b26040 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -165,6 +165,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -192,6 +193,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -224,6 +227,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -277,6 +292,97 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/*
+ * extvac_stats_start() -
+ *
+ * Save extended stats counters before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/*
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -309,6 +415,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -327,7 +435,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -344,6 +452,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -411,6 +520,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -572,6 +683,19 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -586,7 +710,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1378,6 +1503,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2275,11 +2402,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -3120,6 +3249,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3135,6 +3266,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33..d25c9a3679 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index da9a8fe99f..33ff7c81aa 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -691,7 +691,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_vacuum_count(C.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(C.oid) AS autovacuum_count,
pg_stat_get_analyze_count(C.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1381,3 +1383,47 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.pages_frozen AS pages_frozen,
+ stats.pages_all_visible AS pages_all_visible,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.dead_tuples AS dead_tuples,
+
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 86f36b3695..1cff446fb9 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -102,6 +102,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2418,6 +2421,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 4fd6574e12..7f7c7c16e2 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1048,6 +1048,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index ea8c5691e8..3f2e9f5468 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -189,7 +189,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -259,7 +259,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -878,7 +877,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -947,7 +945,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1063,7 +1061,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1113,8 +1111,30 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index faba8b64d2..5dd8275671 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -203,12 +205,37 @@ pgstat_drop_relation(Relation rel)
}
}
+/*
+ * Report an error while vacuuming.
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.errors++;
+ pgstat_unlock_entry(entry_ref);
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -232,6 +259,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -857,6 +886,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -980,3 +1012,36 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+ dst->errors += src->errors;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 60a397dc56..ab3fba3314 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -31,6 +31,42 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+#include "utils/pgstat_internal.h"
+
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -106,6 +142,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_TIMESTAMPTZ(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2069,3 +2111,132 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 22
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_all_visible",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->pages_frozen);
+ values[i++] = Int64GetDatum(extvacuum->pages_all_visible);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 8acca3e0a0..fe554547f5 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd38..aacbbfabae 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12402,4 +12402,22 @@
proargtypes => 'int2',
prosrc => 'gist_stratnum_identity' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables return stats values',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d3..07b28b15d9 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 59c28b4aca..a6f363ac0d 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,6 +169,47 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/*
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a single heap relation.
+ */
+typedef struct ExtVacReport
+{
+ /* number of blocks missed, hit, dirtied and written */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* blocks missed and hit for just the heap */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 errors;
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -209,6 +250,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -267,7 +318,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB1
typedef struct PgStat_ArchiverStats
{
@@ -388,6 +439,7 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
PgStat_Counter parallel_workers_to_launch;
PgStat_Counter parallel_workers_launched;
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
TimestampTz stat_reset_timestamp;
} PgStat_StatDBEntry;
@@ -463,6 +515,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -630,10 +687,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -681,6 +740,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -698,7 +768,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
/*
* Functions in pgstat_replslot.c
*/
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index e54eca5b48..e752c0ce01 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 61b2e1f96b..2c0e55d63f 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -573,7 +573,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
*/
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4d..e93dd4f626 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 3014d047fe..ca7c64f55f 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1804,7 +1804,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_vacuum_count(c.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(c.oid) AS autovacuum_count,
pg_stat_get_analyze_count(c.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -1881,6 +1883,34 @@ pg_stat_database_conflicts| SELECT oid AS datid,
pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock,
pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot
FROM pg_database d;
+pg_stat_get_vacuum_tables| SELECT ns.nspname AS schema,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_gssapi| SELECT pid,
gss_auth AS gss_authenticated,
gss_princ AS principal,
@@ -2188,7 +2218,9 @@ pg_stat_sys_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2236,7 +2268,9 @@ pg_stat_user_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 81e4222d26..977a047202 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-11-13 16:21 Alena Rybakina <[email protected]>
parent: Jim Nasby <[email protected]>
0 siblings, 2 replies; 46+ messages in thread
From: Alena Rybakina @ 2024-11-13 16:21 UTC (permalink / raw)
To: Jim Nasby <[email protected]>; +Cc: Andrei Zubkov <[email protected]>; Alexander Korotkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
Hi! Thank you for your contribution to this thread!
On 13.11.2024 03:24, Jim Nasby wrote:
> On Nov 10, 2024, at 2:09 PM, Alena Rybakina
> <[email protected]> wrote:
>>
>> On 08.11.2024 22:34, Jim Nasby wrote:
>>>
>>>> On Nov 2, 2024, at 7:22 AM, Alena Rybakina
>>>> <[email protected]> wrote:
>>>>
>>>>>> The second is the interrupts field. It is needed for monitoring to know
>>>>>> do we have them or not, so tracking them on the database level will do
>>>>>> the trick. Interrupt is quite rare event, so once the monitoring system
>>>>>> will catch one the DBA can go to the server log for the details.
>>>>> Just to confirm… by “interrupt” you mean vacuum encountered an error?
>>>> Yes it is.
>>> In that case I feel rather strongly that we should label that as
>>> “errors”. “Interrupt” could mean a few different things, but “error”
>>> is very clear.
>>>>
>>>> I updated patches. I excluded system and user time statistics and
>>>> save number of interrupts only for database.I removed the ability
>>>> to get statistics for all tables, now they can only be obtained for
>>>> an oid table [0], as suggested here. I also renamed the statistics
>>>> from pg_stat_vacuum_tables to pg_stat_get_vacuum_tables and
>>>> similarly for indexes and databases. I noticed that that’s what
>>>> they’re mostly called. Ready for discussion.
>>>>
>>> I think it’s better that the views follow the existing naming
>>> conventions (which don’t include “_get_”; only the functions have
>>> that in their names). Assuming that, the only question becomes
>>> pg_stat_vacuum_* vs pg_stat_*_vacuum. Given the existing precedent
>>> of pg_statio_*, I’m inclined to go with pg_stat_vacuum_*.
>> I have fixed it.
>
> I’ve reviewed and made some cosmetic changes to patch 1, though of
> note it looks like an effort has been made to keep
> stat_reset_timestamp at the end of PgStat_StatDBEntry, so I
> re-arranged that. I also removed some obviously dead code. It appears
> that pgstat_update_snapshot(), InitSnapshotIterator() and
> ScanStatSnapshot() are also dead, but I’ve left it in incase I’m
> missing something. The tests are also failing for me because a number
> of psql variables aren’t set.
Thank you! Yes, I have deleted them.
>
> I do think we should separate out the counts for deleted but still
> visible tuples vs tuples where we couldn’t get a cleanup lock (in
> other words, recently_dead_tuples and missed_dead_tuples
> from LVRelState). I realize that’s a departure from how some of the
> existing reporting works, but IMO combining them together isn’t a
> pattern we should be repeating since they mean completely different
> things. Towards that end I did remove missed_dead_tuples from the
> reporting, and renamed ExtVacReport.dead_tuples to
> recently_dead_tuples, but I stopped short of creating a separate entry
> for missed_dead_tuples. Note that while recently_dead_tuples is really
> a global thing (so only needs to be reported at a global (or at most
> per-database) level, but missed_dead_tuples should really be at a
> per-table level.
I am willing to agree with your idea. But we need to think about how
clearly describe them in the documentation.
>
> Updated 0001-v13 attached, as well as the diff between v12 and v13.
Thank you)
And I agree with your changes. And included them in patches.
---
Regards,
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] v13-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (65.0K, 3-v13-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 633c2120eca5774c78818f36b41ecbf47d30b879 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Wed, 13 Nov 2024 16:56:15 +0300
Subject: [PATCH 1/3] Machinery for grabbing an extended vacuum statistics on
heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 140 +++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 48 ++++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 32 ++-
src/backend/utils/activity/pgstat_relation.c | 70 +++++-
src/backend/utils/adt/pgstatfuncs.c | 135 ++++++++++++
src/backend/utils/error/elog.c | 13 ++
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 81 ++++++-
src/include/utils/elog.h | 1 +
src/include/utils/pgstat_internal.h | 2 +-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 51 +++++
src/test/regress/expected/rules.out | 40 +++-
.../expected/vacuum_tables_statistics.out | 203 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 158 ++++++++++++++
22 files changed, 1103 insertions(+), 17 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 793bd33cb4d..12023fd164a 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -165,6 +165,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -192,6 +193,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -224,6 +227,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -277,6 +292,100 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -309,6 +418,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -327,7 +438,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -344,6 +455,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -411,6 +523,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -572,6 +686,19 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -586,7 +713,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1378,6 +1506,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2275,11 +2405,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -3120,6 +3252,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3135,6 +3269,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33c..d25c9a3679d 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index da9a8fe99f2..33ff7c81aa0 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -691,7 +691,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_vacuum_count(C.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(C.oid) AS autovacuum_count,
pg_stat_get_analyze_count(C.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1381,3 +1383,47 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.pages_frozen AS pages_frozen,
+ stats.pages_all_visible AS pages_all_visible,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.dead_tuples AS dead_tuples,
+
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 86f36b36954..1cff446fb92 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -102,6 +102,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2418,6 +2421,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 4fd6574e129..7f7c7c16e23 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1048,6 +1048,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index ea8c5691e87..3f2e9f54685 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -189,7 +189,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -259,7 +259,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -878,7 +877,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -947,7 +945,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1063,7 +1061,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1113,8 +1111,30 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index faba8b64d23..81e41b72f92 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -203,12 +205,40 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.errors++;
+ pgstat_unlock_entry(entry_ref);
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -232,6 +262,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -857,6 +889,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -980,3 +1015,36 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+ dst->errors += src->errors;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->pages_frozen += src->pages_frozen;
+ dst->pages_all_visible += src->pages_all_visible;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 60a397dc561..473af45f194 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_TIMESTAMPTZ(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2069,3 +2075,132 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 22
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_all_visible",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->pages_frozen);
+ values[i++] = Int64GetDatum(extvacuum->pages_all_visible);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 8acca3e0a0b..fe554547f5b 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd382..aacbbfabaeb 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12402,4 +12402,22 @@
proargtypes => 'int2',
prosrc => 'gist_stratnum_identity' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables return stats values',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 59c28b4aca8..bbfd13b2d55 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,6 +169,52 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 errors;
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -209,6 +255,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -267,7 +323,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB1
typedef struct PgStat_ArchiverStats
{
@@ -390,6 +446,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -463,6 +521,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -630,10 +693,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -681,6 +746,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -698,7 +774,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
/*
* Functions in pgstat_replslot.c
*/
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index e54eca5b489..e752c0ce015 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 61b2e1f96b2..2c0e55d63f3 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -573,7 +573,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
*/
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..7cdb79c0ec4
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 100| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 100| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..e93dd4f626c 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5facb2c862c
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,51 @@
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
+# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 3014d047fef..a8a8bffcd4b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1804,7 +1804,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_vacuum_count(c.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(c.oid) AS autovacuum_count,
pg_stat_get_analyze_count(c.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2188,7 +2190,9 @@ pg_stat_sys_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2236,9 +2240,39 @@ pg_stat_user_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schema,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..9071539dddc
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,203 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | pages_frozen | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+--------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ 0 | 0 | 0 | 0
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | t | t
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ f | f | t | t
+(1 row)
+
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' \gset
+ERROR: column "rev_all_frozen_pages" does not exist
+LINE 1: ...LECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_fr...
+ ^
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
+--------------+-------------------+----------------------+-----------------------
+ t | t | f | f
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 81e4222d26a..977a0472027 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..0463973ce0b
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,158 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,pages_frozen,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,pages_frozen > 0 AS pages_frozen,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,pages_frozen-:fp > 0 AS pages_frozen,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT pages_frozen AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT pages_frozen > 0 AS pages_frozen,pages_all_visible > 0 AS pages_all_visible,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+DROP TABLE vestat CASCADE;
\ No newline at end of file
--
2.34.1
[text/x-patch] v13-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (43.2K, 4-v13-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From c99ef011750f47c7a1ca6efc62a462b54dd5bbe7 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Wed, 13 Nov 2024 16:57:22 +0300
Subject: [PATCH 2/3] Machinery for grabbing an extended vacuum statistics on
heap and index relations. Remember, statistic on heap and index relations a
bit different (see ExtVacReport to find out more information). The concept of
the ExtVacReport structure has been complicated to store statistic
information for two kinds of relations: for heap and index relations.
ExtVacReportType variable helps to determine what the kind is considering
now.
---
src/backend/access/heap/vacuumlazy.c | 99 +++++++++--
src/backend/catalog/system_views.sql | 32 ++++
src/backend/utils/activity/pgstat.c | 7 +-
src/backend/utils/activity/pgstat_relation.c | 41 +++--
src/backend/utils/adt/pgstatfuncs.c | 126 ++++++++++++-
src/include/catalog/pg_proc.dat | 9 +
src/include/pgstat.h | 52 ++++--
.../vacuum-extending-in-repetable-read.out | 7 +-
src/test/regress/expected/rules.out | 22 +++
.../expected/vacuum_index_statistics.out | 165 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 14 +-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 130 ++++++++++++++
.../regress/sql/vacuum_tables_statistics.sql | 10 +-
14 files changed, 661 insertions(+), 54 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 12023fd164a..5a844acf878 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -166,6 +166,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -240,6 +241,13 @@ typedef struct LVExtStatCounters
PgStat_Counter blocks_hit;
} LVExtStatCounters;
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -387,6 +395,46 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+static void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+static void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -690,14 +738,15 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
extvac_stats_end(rel, &extVacCounters, &extVacReport);
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.pages_frozen = vacrel->set_frozen_pages;
- extVacReport.pages_all_visible = vacrel->set_all_visible_pages;
- extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.type = PGSTAT_EXTVAC_HEAP;
+ extVacReport.heap.pages_scanned = vacrel->scanned_pages;
+ extVacReport.heap.pages_removed = vacrel->removed_pages;
+ extVacReport.heap.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.heap.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.heap.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.heap.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.heap.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.heap.index_vacuum_count = vacrel->num_index_scans;
/*
* Report results to the cumulative stats system, too.
@@ -2562,6 +2611,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2580,6 +2633,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2588,6 +2642,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, (void *) vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2612,6 +2673,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2631,12 +2696,20 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3253,7 +3326,7 @@ vacuum_error_callback(void *arg)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3270,7 +3343,7 @@ vacuum_error_callback(void *arg)
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
if(geterrelevel() == ERROR)
- pgstat_report_vacuum_error(errinfo->reloid);
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3286,16 +3359,22 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 33ff7c81aa0..bebf793514a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1427,3 +1427,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_get_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 3f2e9f54685..551dcfa3198 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1124,7 +1124,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
PG_TRY();
{
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
- pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ if (kind == PGSTAT_KIND_RELATION)
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
}
PG_FINALLY();
{
@@ -1179,6 +1180,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 81e41b72f92..c9238e3f650 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -212,7 +212,7 @@ pgstat_drop_relation(Relation rel)
* ---------
*/
void
-pgstat_report_vacuum_error(Oid tableoid)
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -229,6 +229,7 @@ pgstat_report_vacuum_error(Oid tableoid)
tabentry = &shtabentry->stats;
tabentry->vacuum_ext.errors++;
+ tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
}
@@ -1036,15 +1037,31 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->pages_frozen += src->pages_frozen;
- dst->pages_all_visible += src->pages_all_visible;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.recently_dead_tuples += src->heap.recently_dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 473af45f194..f797cf2d7f3 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2175,14 +2175,14 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->pages_frozen);
- values[i++] = Int64GetDatum(extvacuum->pages_all_visible);
- values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_frozen);
+ values[i++] = Int64GetDatum(extvacuum->heap.pages_all_visible);
+ values[i++] = Int64GetDatum(extvacuum->heap.tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->heap.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->heap.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->heap.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2201,6 +2201,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->index.tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index aacbbfabaeb..be8a10533bb 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12420,4 +12420,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index bbfd13b2d55..a47debdc351 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -169,11 +169,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_HEAP = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -205,14 +213,38 @@ typedef struct ExtVacReport
/* Interruptions on any errors. */
int32 errors;
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 pages_frozen; /* pages marked in VM as frozen */
- int64 pages_all_visible; /* pages marked in VM as all-visible */
- int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
- int64 index_vacuum_count; /* number of index vacuumings */
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } heap;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -698,7 +730,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
-extern void pgstat_report_vacuum_error(Oid tableoid);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 7cdb79c0ec4..93fe15c01f9 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -9,10 +9,9 @@ step s2_print_vacuum_stats_table:
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
-relname |tuples_deleted|dead_tuples|tuples_frozen
---------------------------+--------------+-----------+-------------
-test_vacuum_stat_isolation| 0| 0| 0
-(1 row)
+relname|tuples_deleted|dead_tuples|tuples_frozen
+-------+--------------+-----------+-------------
+(0 rows)
step s1_begin_repeatable_read:
BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a8a8bffcd4b..bede88721c0 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1883,6 +1883,28 @@ pg_stat_database_conflicts| SELECT oid AS datid,
pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock,
pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot
FROM pg_database d;
+pg_stat_get_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_gssapi| SELECT pid,
gss_auth AS gss_authenticated,
gss_princ AS principal,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..3a1ae648d0e
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,165 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+SELECT min(relid) FROM pg_stat_get_vacuum_indexes(0);
+ min
+-----
+ 0
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index 9071539dddc..eee468a50d5 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -23,8 +23,6 @@ SELECT pg_stat_force_next_flush();
(1 row)
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -156,6 +154,14 @@ WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
vestat | t | t | f | t | t
(1 row)
+DROP TABLE vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
-- must be empty
@@ -183,7 +189,7 @@ SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_v
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
--------------+-------------------+----------------------+-----------------------
- f | f | t | t
+ f | f | f | f
(1 row)
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
@@ -197,7 +203,7 @@ SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_v
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
pages_frozen | pages_all_visible | rev_all_frozen_pages | rev_all_visible_pages
--------------+-------------------+----------------------+-----------------------
- t | t | f | f
+ t | t | t | t
(1 row)
DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 977a0472027..9847a330ed1 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..e3cddee6601
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,130 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_get_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_get_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+SELECT min(relid) FROM pg_stat_get_vacuum_indexes(0);
+
+DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
index 0463973ce0b..ee74bb3a958 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -17,8 +17,7 @@ SET track_functions TO 'all';
SELECT pg_stat_force_next_flush();
\set sample_size 10000
-SET vacuum_freeze_min_age = 0;
-SET vacuum_freeze_table_age = 0;
+
--SET stats_fetch_consistency = snapshot;
CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
@@ -126,13 +125,16 @@ SELECT vt.relname,pages_frozen-:fp = 0 AS pages_frozen,tuples_deleted-:td = 0 AS
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+DROP TABLE vestat;
+SELECT pg_stat_force_next_flush();
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
-- must be empty
SELECT pages_frozen, pages_all_visible, rev_all_frozen_pages,rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
-
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- backend defreezed pages
@@ -143,14 +145,12 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
UPDATE vestat SET x = x + 1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' \gset
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-
-- vacuum freezed pages
SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
--
2.34.1
[text/x-patch] v13-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (22.2K, 5-v13-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From f746fcc6af12d557189bc4d53e4aa8e3a4f87e31 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Wed, 13 Nov 2024 16:58:16 +0300
Subject: [PATCH 3/3] Machinery for grabbing an extended vacuum statistics on
databases. It transmits vacuum statistical information about each table and
accumulates it for the database which the table belonged.
---
src/backend/catalog/system_views.sql | 26 ++++-
src/backend/utils/activity/pgstat.c | 2 +
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 16 ++++
src/backend/utils/adt/pgstatfuncs.c | 95 +++++++++++++++++++
src/include/catalog/pg_proc.dat | 15 ++-
src/include/pgstat.h | 3 +-
src/test/regress/expected/rules.out | 16 ++++
...ut => vacuum_tables_and_db_statistics.out} | 81 +++++++++++++++-
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 66 ++++++++++++-
11 files changed, 309 insertions(+), 14 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (77%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (79%)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index bebf793514a..f337768ea55 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1458,4 +1458,28 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_get_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.errors AS errors
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 551dcfa3198..5b81fbba12a 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1126,6 +1126,8 @@ pgstat_update_snapshot(PgStat_Kind kind)
pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
if (kind == PGSTAT_KIND_RELATION)
pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ else if (kind == PGSTAT_KIND_DATABASE)
+ pgstat_build_snapshot(PGSTAT_KIND_DATABASE);
}
PG_FINALLY();
{
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 7757d2ace74..840d848a752 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -449,6 +449,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index c9238e3f650..853ee510bb5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -218,6 +218,7 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
if (!pgstat_track_counts)
return;
@@ -231,6 +232,10 @@ pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
tabentry->vacuum_ext.errors++;
tabentry->vacuum_ext.type = m_type;
pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.errors++;
+ dbentry->vacuum_ext.type = m_type;
}
/*
@@ -244,6 +249,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -297,6 +303,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index f797cf2d7f3..32e38d43062 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2311,6 +2311,101 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 13
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index be8a10533bb..e12d77f1e3b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12403,7 +12403,7 @@
prosrc => 'gist_stratnum_identity' },
{ oid => '8001',
- descr => 'pg_stat_get_vacuum_tables return stats values',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
@@ -12421,12 +12421,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index a47debdc351..6271d96b415 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -174,7 +174,8 @@ typedef enum ExtVacReportType
{
PGSTAT_EXTVAC_INVALID = 0,
PGSTAT_EXTVAC_HEAP = 1,
- PGSTAT_EXTVAC_INDEX = 2
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
} ExtVacReportType;
/* ----------
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index bede88721c0..05be0949871 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1883,6 +1883,22 @@ pg_stat_database_conflicts| SELECT oid AS datid,
pg_stat_get_db_conflict_startup_deadlock(oid) AS confl_deadlock,
pg_stat_get_db_conflict_logicalslot(oid) AS confl_active_logicalslot
FROM pg_database d;
+pg_stat_get_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, errors);
pg_stat_get_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 77%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index eee468a50d5..8468f7edaeb 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,6 +6,9 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
-- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
@@ -183,7 +186,7 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
-UPDATE vestat SET x = x + 1001;
+UPDATE vestat SET x = x+1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
@@ -193,10 +196,7 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
(1 row)
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
-FROM pg_stat_vacuum_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' \gset
-ERROR: column "rev_all_frozen_pages" does not exist
-LINE 1: ...LECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_fr...
- ^
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- vacuum freezed pages
SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
@@ -206,4 +206,75 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
t | t | t | t
(1 row)
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
DROP TABLE vestat CASCADE;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 9847a330ed1..1ba32b87cf5 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 79%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index ee74bb3a958..47df1056ad8 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,6 +7,10 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+
-- conditio sine qua non
SHOW track_counts; -- must be on
-- not enabled by default, but we want to test it...
@@ -143,16 +147,72 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
-UPDATE vestat SET x = x + 1001;
+UPDATE vestat SET x = x+1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
SELECT pages_frozen > :pf AS pages_frozen,pages_all_visible > :pv AS pages_all_visible,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
SELECT pages_frozen AS pf, pages_all_visible AS pv, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
-FROM pg_stat_vacuum_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' \gset
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
-- vacuum freezed pages
SELECT pages_frozen = :pf AS pages_frozen,pages_all_visible = :pv AS pages_all_visible,rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
-DROP TABLE vestat CASCADE;
\ No newline at end of file
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = current_database();
+
+DROP TABLE vestat CASCADE;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_get_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_database(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
\ No newline at end of file
--
2.34.1
[text/x-patch] v13-0004-Add-documentation-about-the-system-views-that-are-us.patch (27.1K, 6-v13-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From 4134c047c63f2b6acc3fe119f7fbd3d66c085bea Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sun, 25 Aug 2024 17:47:55 +0300
Subject: [PATCH] Add documentation about the system views that are used in the
machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 747 ++++++++++++++++++
.../vacuum-extending-in-repetable-read.out | 6 +-
.../vacuum-extending-in-repetable-read.spec | 4 +-
3 files changed, 752 insertions(+), 5 deletions(-)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 61d28e701f2..b74c53bb000 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5064,4 +5064,751 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-get-vacuum-database">
+ <title><structname>pg_stat_get_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-database">
+ <primary>pg_stat_get_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-get-vacuum-indexes">
+ <title><structname>pg_stat_get_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-indexes">
+ <primary>pg_stat_get_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this index
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-get-vacuum-tables">
+ <title><structname>pg_stat_get_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-tables">
+ <primary>pg_stat_get_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-frozen in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_all_visible</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times vacuum operations marked pages of this table
+ as all-visible in the visibility map
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the visibility map
+ was removed for pages of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>interrupts</structfield> <type>float8</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this table
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
+
</chapter>
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
index 93fe15c01f9..a7794023508 100644
--- a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -6,7 +6,7 @@ step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, iv
step s2_print_vacuum_stats_table:
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
relname|tuples_deleted|dead_tuples|tuples_frozen
@@ -28,7 +28,7 @@ step s2_vacuum: VACUUM test_vacuum_stat_isolation;
step s2_print_vacuum_stats_table:
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
relname |tuples_deleted|dead_tuples|tuples_frozen
@@ -42,7 +42,7 @@ step s2_vacuum: VACUUM test_vacuum_stat_isolation;
step s2_print_vacuum_stats_table:
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
relname |tuples_deleted|dead_tuples|tuples_frozen
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
index 5facb2c862c..6e33df46480 100644
--- a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -1,4 +1,4 @@
-# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stat_get_vacuum_tables.
# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
# by the value of the cleared tuples that the vacuum managed to clear.
@@ -33,7 +33,7 @@ step s2_print_vacuum_stats_table
{
SELECT
vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
- FROM pg_stat_vacuum_tables vt, pg_class c
+ FROM pg_stat_get_vacuum_tables vt, pg_class c
WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
}
--
2.34.1
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-11-30 04:48 Kirill Reshke <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 1 reply; 46+ messages in thread
From: Kirill Reshke @ 2024-11-30 04:48 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Alexander Korotkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
On Wed, 13 Nov 2024 at 21:21, Alena Rybakina <[email protected]> wrote:
>
> Hi! Thank you for your contribution to this thread!
>
> On 13.11.2024 03:24, Jim Nasby wrote:
>
> On Nov 10, 2024, at 2:09 PM, Alena Rybakina <[email protected]> wrote:
>
>
> On 08.11.2024 22:34, Jim Nasby wrote:
>
>
> On Nov 2, 2024, at 7:22 AM, Alena Rybakina <[email protected]> wrote:
>
> The second is the interrupts field. It is needed for monitoring to know
> do we have them or not, so tracking them on the database level will do
> the trick. Interrupt is quite rare event, so once the monitoring system
> will catch one the DBA can go to the server log for the details.
>
> Just to confirm… by “interrupt” you mean vacuum encountered an error?
>
> Yes it is.
>
> In that case I feel rather strongly that we should label that as “errors”. “Interrupt” could mean a few different things, but “error” is very clear.
>
> I updated patches. I excluded system and user time statistics and save number of interrupts only for database. I removed the ability to get statistics for all tables, now they can only be obtained for an oid table [0], as suggested here. I also renamed the statistics from pg_stat_vacuum_tables to pg_stat_get_vacuum_tables and similarly for indexes and databases. I noticed that that’s what they’re mostly called. Ready for discussion.
>
> I think it’s better that the views follow the existing naming conventions (which don’t include “_get_”; only the functions have that in their names). Assuming that, the only question becomes pg_stat_vacuum_* vs pg_stat_*_vacuum. Given the existing precedent of pg_statio_*, I’m inclined to go with pg_stat_vacuum_*.
>
> I have fixed it.
>
>
> I’ve reviewed and made some cosmetic changes to patch 1, though of note it looks like an effort has been made to keep stat_reset_timestamp at the end of PgStat_StatDBEntry, so I re-arranged that. I also removed some obviously dead code. It appears that pgstat_update_snapshot(), InitSnapshotIterator() and ScanStatSnapshot() are also dead, but I’ve left it in incase I’m missing something. The tests are also failing for me because a number of psql variables aren’t set.
>
> Thank you! Yes, I have deleted them.
>
>
> I do think we should separate out the counts for deleted but still visible tuples vs tuples where we couldn’t get a cleanup lock (in other words, recently_dead_tuples and missed_dead_tuples from LVRelState). I realize that’s a departure from how some of the existing reporting works, but IMO combining them together isn’t a pattern we should be repeating since they mean completely different things. Towards that end I did remove missed_dead_tuples from the reporting, and renamed ExtVacReport.dead_tuples to recently_dead_tuples, but I stopped short of creating a separate entry for missed_dead_tuples. Note that while recently_dead_tuples is really a global thing (so only needs to be reported at a global (or at most per-database) level, but missed_dead_tuples should really be at a per-table level.
>
> I am willing to agree with your idea. But we need to think about how clearly describe them in the documentation.
>
>
> Updated 0001-v13 attached, as well as the diff between v12 and v13.
>
> Thank you)
>
> And I agree with your changes. And included them in patches.
>
> ---
> Regards,
> Alena Rybakina
> Postgres Professional
Hello!
After a brief glance, I think this patch set is good.
But there isn't any more time in the current CF to commit this :(.
So I moved to the next CF.
I also like the 0001 commit message. This commit message is quite
large and easy to understand. Actually, it might be too big. Perhaps
rather of being a commit message, the final paragraph (pages_frozen -
number of pages that..) need to be a part of the document. Perhaps
delete the explanation on pages_frozen that we have in 0004?
--
Best regards,
Kirill Reshke
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-12-02 08:27 Alexander Korotkov <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 2 replies; 46+ messages in thread
From: Alexander Korotkov @ 2024-12-02 08:27 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
Hi, Alena!
On Wed, Nov 13, 2024 at 6:21 PM Alena Rybakina
<[email protected]> wrote:
> Updated 0001-v13 attached, as well as the diff between v12 and v13.
>
> Thank you)
>
> And I agree with your changes. And included them in patches.
Thank you for the updated patchset. Some points from me.
* I've read the previous discussion on how important to keep all these
fields regarding vacuum statistics including points by Andrei and Jim.
It still worrying me that statistics volume is going to burst in about
3 times, but I don't have a particular proposal on how to make more
granular approach. I wonder if you could propose something.
* Previously PGSTAT_FILE_FORMAT_ID got increased by 1. Your 0001 patch
increases it by 2. It's minor note, but I'd like to keep the
tradition.
* Commit message for 0001 looks nice, but commit messages of 0002,
0003, and 0004 look messy. Could you please, rearrange them.
* The distinction between 0001 and 0002 is not clear. The first line
of 0001 is "Machinery for grabbing an extended vacuum statistics on
heap relations", the first line of 0002 is "Machinery for grabbing an
extended vacuum statistics on heap and index relations." I guess 0001
should be about heap relations while 0002 should be about just index
relations. Is this correct?
* I guess this statistics should work for any table AM, based on what
has been done in relation_vacuum() interface method. If that's
correct, we need to get rid of "heap" terminology and use "table"
instead.
* 0004 should be pure documentation patch, but it seems containing
changes to isolation tests. Please, move them into a more appropriate
place.
------
Regards,
Alexander Korotkov
Supabase
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-12-02 14:46 Ilia Evdokimov <[email protected]>
parent: Alexander Korotkov <[email protected]>
1 sibling, 2 replies; 46+ messages in thread
From: Ilia Evdokimov @ 2024-12-02 14:46 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
In my opinion, the patches are semantically correct. However, not all
dead code has been removed - I'm referring to pgstat_update_snapshot().
Also, the tests need to be fixed.
--
Best regards,
Ilia Evdokimov,
Tantor Labs LLC.
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-12-02 20:12 Alena Rybakina <[email protected]>
parent: Alexander Korotkov <[email protected]>
1 sibling, 2 replies; 46+ messages in thread
From: Alena Rybakina @ 2024-12-02 20:12 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
On 02.12.2024 11:27, Alexander Korotkov wrote:
> Hi, Alena!
>
> On Wed, Nov 13, 2024 at 6:21 PM Alena Rybakina
> <[email protected]> wrote:
>> Updated 0001-v13 attached, as well as the diff between v12 and v13.
>>
>> Thank you)
>>
>> And I agree with your changes. And included them in patches.
> Thank you for the updated patchset. Some points from me.
>
> * I've read the previous discussion on how important to keep all these
> fields regarding vacuum statistics including points by Andrei and Jim.
> It still worrying me that statistics volume is going to burst in about
> 3 times, but I don't have a particular proposal on how to make more
> granular approach. I wonder if you could propose something.
> * Previously PGSTAT_FILE_FORMAT_ID got increased by 1. Your 0001 patch
> increases it by 2. It's minor note, but I'd like to keep the
> tradition.
> * Commit message for 0001 looks nice, but commit messages of 0002,
> 0003, and 0004 look messy. Could you please, rearrange them.
> * The distinction between 0001 and 0002 is not clear. The first line
> of 0001 is "Machinery for grabbing an extended vacuum statistics on
> heap relations", the first line of 0002 is "Machinery for grabbing an
> extended vacuum statistics on heap and index relations." I guess 0001
> should be about heap relations while 0002 should be about just index
> relations. Is this correct?
> * I guess this statistics should work for any table AM, based on what
> has been done in relation_vacuum() interface method. If that's
> correct, we need to get rid of "heap" terminology and use "table"
> instead.
> * 0004 should be pure documentation patch, but it seems containing
> changes to isolation tests. Please, move them into a more appropriate
> place.
>
Thank you for your valuable feedback, I am already carefully processing
your comments and will update the patches soon.
I will think about what can be done to address the problem of increasing
the volume of statistics; perhaps it will be possible to implement a guc
that, when enabled, will accumulate additional information on vacuum
statistics. For example, this way you can group statistics by buffers
and vacuum statistics.
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-12-02 20:59 Alena Rybakina <[email protected]>
parent: Kirill Reshke <[email protected]>
0 siblings, 0 replies; 46+ messages in thread
From: Alena Rybakina @ 2024-12-02 20:59 UTC (permalink / raw)
To: Kirill Reshke <[email protected]>; +Cc: Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Alexander Korotkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
Hi! Thank you for your review!
On 30.11.2024 07:48, Kirill Reshke wrote:
> Hello!
> After a brief glance, I think this patch set is good.
> But there isn't any more time in the current CF to commit this :(.
> So I moved to the next CF.
I agree with you. Thank you!)
> I also like the 0001 commit message. This commit message is quite
> large and easy to understand. Actually, it might be too big. Perhaps
> rather of being a commit message, the final paragraph (pages_frozen -
> number of pages that..) need to be a part of the document. Perhaps
> delete the explanation on pages_frozen that we have in 0004?
To be honest, I don't quite understand what you're suggesting. Are you
suggesting moving the explanation about the pages_frozen from the commit
message to the documentation or fixing something in the documentation
about the pages_frozen? Can you please explain?
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-12-02 21:00 Alena Rybakina <[email protected]>
parent: Ilia Evdokimov <[email protected]>
1 sibling, 0 replies; 46+ messages in thread
From: Alena Rybakina @ 2024-12-02 21:00 UTC (permalink / raw)
To: Ilia Evdokimov <[email protected]>; +Cc: Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
On 02.12.2024 17:46, Ilia Evdokimov wrote:
> In my opinion, the patches are semantically correct. However, not all
> dead code has been removed - I'm referring to
> pgstat_update_snapshot(). Also, the tests need to be fixed.
>
Thank you, I'll fix it
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-12-19 10:37 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2024-12-19 10:37 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; Jim Nasby <[email protected]>; Ilia Evdokimov <[email protected]>; Kirill Reshke <[email protected]>; +Cc: Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
Hi! I updated patch.
On 02.12.2024 23:12, Alena Rybakina wrote:
> On 02.12.2024 11:27, Alexander Korotkov wrote:
>> Hi, Alena!
>>
>> On Wed, Nov 13, 2024 at 6:21 PM Alena Rybakina
>> <[email protected]> wrote:
>>> Updated 0001-v13 attached, as well as the diff between v12 and v13.
>>>
>>> Thank you)
>>>
>>> And I agree with your changes. And included them in patches.
>> Thank you for the updated patchset. Some points from me.
>>
>> * I've read the previous discussion on how important to keep all these
>> fields regarding vacuum statistics including points by Andrei and Jim.
>> It still worrying me that statistics volume is going to burst in about
>> 3 times, but I don't have a particular proposal on how to make more
>> granular approach. I wonder if you could propose something.
I propose to solve this with a guc (track_vacuum_statistics) that will
allow us to disable statistics collection or enable it back when needed.
This statistics is useful if you collect it over a certain period of
time and watch the dynamics of the change, so I think a hook will not
hurt here.
I also added new statistics vm_new_visible_frozen_pages due to the
dc6acfd commit and renamed some of my statistics from frozen_pages to
vm_new_frozen_pages and all-visible_pages to vm_new_visible_pages. I
also added statistics missed_tupes, missed_pages. Both are necessary to
take into account how many tuples were not cleared by vacuum due to
failure to acquire a cleanup lock on a heap page. The second statistics
missed_pages will allow you to track whether this is one particular page
or not.
This information can make it clear that perhaps the data is broken
somewhere or there is an error in the operation of the database, for
example.
>> * Previously PGSTAT_FILE_FORMAT_ID got increased by 1. Your 0001 patch
>> increases it by 2. It's minor note, but I'd like to keep the
>> tradition.
Fixed
>> * Commit message for 0001 looks nice, but commit messages of 0002,
>> 0003, and 0004 look messy. Could you please, rearrange them.
Fixed
>> * The distinction between 0001 and 0002 is not clear. The first line
>> of 0001 is "Machinery for grabbing an extended vacuum statistics on
>> heap relations", the first line of 0002 is "Machinery for grabbing an
>> extended vacuum statistics on heap and index relations." I guess 0001
>> should be about heap relations while 0002 should be about just index
>> relations. Is this correct?
Fixed
>> * I guess this statistics should work for any table AM, based on what
>> has been done in relation_vacuum() interface method. If that's
>> correct, we need to get rid of "heap" terminology and use "table"
>> instead.
Fixed
>> * 0004 should be pure documentation patch, but it seems containing
>> changes to isolation tests. Please, move them into a more appropriate
>> place.
>>
Fixed
Thanks for your review, it was very helpful)
I also noticed that my stats for indexes were not being collected while
parallel vacuum was running. I fixed it by adding some extra code that
basically captured changes to the parallel_vacuum_process_all_indexes
function. I used a script like this to check if everything was correct.
pgbench -d postgres -i -s 10
my/inst/bin/pg_basebackup -D ~/backup
#psql
--check parallel vacuum statistics
create index accounts_idx1 on pgbench_accounts(bid);
create index accounts_idx2 on pgbench_accounts(aid, bid);
delete from pgbench_accounts where aid >5;
set max_parallel_maintenance_workers = 8;
VACUUM (PARALLEL 3) pgbench_accounts;
create index accounts_idx1 on pgbench_accounts(bid);
create index accounts_idx2 on pgbench_accounts(aid, bid);
delete from pgbench_accounts where aid >5;
set max_parallel_maintenance_workers = 8;
VACUUM (PARALLEL 3) pgbench_accounts;
pg_ctl -D ../postgres_data11 -l logfile stop
rm -rf ../postgres_data/*
cp -r ~/backup/* ~/postgres_data/
pg_ctl -D ../postgres_data11 -l logfile start
--check vacuum statistics processed by postmaster only
create index accounts_idx1 on pgbench_accounts(bid);
create index accounts_idx2 on pgbench_accounts(aid, bid);
delete from pgbench_accounts where aid >5;
set max_parallel_maintenance_workers = 8;
VACUUM (PARALLEL 0) pgbench_accounts;
To view statistics:
select vt.relname, total_blks_read AS total_blks_read,
total_blks_hit AS total_blks_hit,
total_blks_dirtied AS total_blks_dirtied,
total_blks_written AS total_blks_written,
rel_blks_read AS rel_blks_read,
rel_blks_hit AS rel_blks_hit,
pages_deleted AS pages_deleted,
tuples_deleted AS tuples_deleted,
wal_records AS wal_records,
wal_fpi AS wal_fpi,
wal_bytes AS wal_bytes,
blk_read_time AS blk_read_time,
blk_write_time AS blk_write_time,
delay_time AS delay_time,
total_time AS total_time
FROM pg_stat_get_vacuum_indexes vt, pg_class c
WHERE (vt.relname='accounts_idx1' or vt.relname='accounts_idx2' or
vt.relname = 'pgbench_accounts_pkey') AND vt.relid = c.oid;
select stats.relname,stats.total_blks_read AS total_blks_read,
stats.total_blks_hit AS total_blks_hit,
stats.total_blks_dirtied AS total_blks_dirtied,
stats.total_blks_written AS total_blks_written,
stats.rel_blks_read AS rel_blks_read,
stats.rel_blks_hit AS rel_blks_hit,
stats.pages_scanned AS pages_scanned,
stats.pages_removed AS pages_removed,
stats.pages_frozen AS pages_frozen,
stats.pages_all_visible AS pages_all_visible,
stats.tuples_deleted AS tuples_deleted,
stats.tuples_frozen AS tuples_frozen,
stats.dead_tuples AS dead_tuples,
stats.index_vacuum_count AS index_vacuum_count,
stats.wal_records AS wal_records,
stats.wal_fpi AS wal_fpi,
stats.wal_bytes AS wal_bytes,
stats.blk_read_time AS blk_read_time,
stats.blk_write_time AS blk_write_time,
stats.delay_time AS delay_time,
stats.total_time AS total_time from pg_stat_vacuum_tables stats,
pg_stat_all_tables WHERE stats.relname = 'pgbench_accounts' and
stats.relid = pg_stat_all_tables.relid;
output_single_19I got the following results and stored them in
output_single_19 and output_parallel_19 files.
I noticed that rel_blks_read and rel_blks_hit are too small compared to
the vacuum statistics when the vacuum is not parallel. I suspect that
this problem is related to the fact that the relationship statistics
have not reached that time. You can see that they are calculated in my
patch like this:
report->blks_fetched =
rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
report->blks_hit =
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
The second thing that bothered me was that some table stats differ in
the fields total_blks_read, rel_blks_read, pages_removed. If with the
buffer this could be related to the fact that in a single run we rely on
the stats of the global buffer and shaft statistics and this could
explain why there are more of them, then with pages_removed I have no
explanation yet as to what could have happened. I am still studying this.
When you have time, take a look at the patches, I will be glad to
receive any feedback.
--
Regards,
Alena Rybakina
Postgres Professional
-[ RECORD 1 ]--------+-----------------
relname | pgbench_accounts
total_blks_read | 15212
total_blks_hit | 102455
total_blks_dirtied | 13721
total_blks_written | 13458
rel_blks_read | 8872
rel_blks_hit | 51957
pages_scanned | 16394
pages_removed | 11632
tuples_deleted | 999934
tuples_frozen | 0
recently_dead_tuples | 0
index_vacuum_count | 1
wal_records | 68110
wal_fpi | 2754
wal_bytes | 32363402
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 949.217
-[ RECORD 1 ]------+----------------------
relname | pgbench_accounts_pkey
total_blks_read | 2745
total_blks_hit | 21871
total_blks_dirtied | 2745
total_blks_written | 2632
rel_blks_read | 2745
rel_blks_hit | 21871
pages_deleted | 2735
tuples_deleted | 999934
wal_records | 8197
wal_fpi | 2744
wal_bytes | 22799390
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 247.65800000000002
-[ RECORD 2 ]------+----------------------
relname | accounts_idx1
total_blks_read | 850
total_blks_hit | 6726
total_blks_dirtied | 850
total_blks_written | 750
rel_blks_read | 850
rel_blks_hit | 6726
pages_deleted | 840
tuples_deleted | 999934
wal_records | 2522
wal_fpi | 0
wal_bytes | 202130
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 79.552
-[ RECORD 3 ]------+----------------------
relname | accounts_idx2
total_blks_read | 2745
total_blks_hit | 21871
total_blks_dirtied | 2745
total_blks_written | 2673
rel_blks_read | 2745
rel_blks_hit | 21871
pages_deleted | 2735
tuples_deleted | 999934
wal_records | 8197
wal_fpi | 0
wal_bytes | 2606548
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 143.95000000000002
-[ RECORD 1 ]------+----------------------
relname | pgbench_accounts_pkey
total_blks_read | 2745
total_blks_hit | 21894
total_blks_dirtied | 2747
total_blks_written | 2640
rel_blks_read | 2745
rel_blks_hit | 21886
pages_deleted | 2738
tuples_deleted | 999995
wal_records | 8205
wal_fpi | 2744
wal_bytes | 22816484
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 249.222
-[ RECORD 2 ]------+----------------------
relname | accounts_idx1
total_blks_read | 851
total_blks_hit | 6749
total_blks_dirtied | 852
total_blks_written | 849
rel_blks_read | 0
rel_blks_hit | 1
pages_deleted | 843
tuples_deleted | 999995
wal_records | 2530
wal_fpi | 0
wal_bytes | 202834
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 249.176
-[ RECORD 3 ]------+----------------------
relname | accounts_idx2
total_blks_read | 2746
total_blks_hit | 21894
total_blks_dirtied | 2747
total_blks_written | 2691
rel_blks_read | 0
rel_blks_hit | 1
pages_deleted | 2738
tuples_deleted | 999995
wal_records | 8205
wal_fpi | 0
wal_bytes | 2607518
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 249.174
-[ RECORD 1 ]--------+------------------
relname | pgbench_accounts
total_blks_read | 20513
total_blks_hit | 102219
total_blks_dirtied | 13973
total_blks_written | 13870
rel_blks_read | 14173
rel_blks_hit | 51664
pages_scanned | 16452
pages_removed | 16451
tuples_deleted | 999995
tuples_frozen | 0
recently_dead_tuples | 0
index_vacuum_count | 1
wal_records | 68136
wal_fpi | 2759
wal_bytes | 32422971
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 773.4699999999999
Attachments:
[text/plain] output_single_19 (2.0K, 3-output_single_19)
download | inline:
-[ RECORD 1 ]--------+-----------------
relname | pgbench_accounts
total_blks_read | 15212
total_blks_hit | 102455
total_blks_dirtied | 13721
total_blks_written | 13458
rel_blks_read | 8872
rel_blks_hit | 51957
pages_scanned | 16394
pages_removed | 11632
tuples_deleted | 999934
tuples_frozen | 0
recently_dead_tuples | 0
index_vacuum_count | 1
wal_records | 68110
wal_fpi | 2754
wal_bytes | 32363402
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 949.217
-[ RECORD 1 ]------+----------------------
relname | pgbench_accounts_pkey
total_blks_read | 2745
total_blks_hit | 21871
total_blks_dirtied | 2745
total_blks_written | 2632
rel_blks_read | 2745
rel_blks_hit | 21871
pages_deleted | 2735
tuples_deleted | 999934
wal_records | 8197
wal_fpi | 2744
wal_bytes | 22799390
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 247.65800000000002
-[ RECORD 2 ]------+----------------------
relname | accounts_idx1
total_blks_read | 850
total_blks_hit | 6726
total_blks_dirtied | 850
total_blks_written | 750
rel_blks_read | 850
rel_blks_hit | 6726
pages_deleted | 840
tuples_deleted | 999934
wal_records | 2522
wal_fpi | 0
wal_bytes | 202130
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 79.552
-[ RECORD 3 ]------+----------------------
relname | accounts_idx2
total_blks_read | 2745
total_blks_hit | 21871
total_blks_dirtied | 2745
total_blks_written | 2673
rel_blks_read | 2745
rel_blks_hit | 21871
pages_deleted | 2735
tuples_deleted | 999934
wal_records | 8197
wal_fpi | 0
wal_bytes | 2606548
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 143.95000000000002
[text/plain] output_parallel_19 (2.0K, 4-output_parallel_19)
download | inline:
-[ RECORD 1 ]------+----------------------
relname | pgbench_accounts_pkey
total_blks_read | 2745
total_blks_hit | 21894
total_blks_dirtied | 2747
total_blks_written | 2640
rel_blks_read | 2745
rel_blks_hit | 21886
pages_deleted | 2738
tuples_deleted | 999995
wal_records | 8205
wal_fpi | 2744
wal_bytes | 22816484
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 249.222
-[ RECORD 2 ]------+----------------------
relname | accounts_idx1
total_blks_read | 851
total_blks_hit | 6749
total_blks_dirtied | 852
total_blks_written | 849
rel_blks_read | 0
rel_blks_hit | 1
pages_deleted | 843
tuples_deleted | 999995
wal_records | 2530
wal_fpi | 0
wal_bytes | 202834
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 249.176
-[ RECORD 3 ]------+----------------------
relname | accounts_idx2
total_blks_read | 2746
total_blks_hit | 21894
total_blks_dirtied | 2747
total_blks_written | 2691
rel_blks_read | 0
rel_blks_hit | 1
pages_deleted | 2738
tuples_deleted | 999995
wal_records | 8205
wal_fpi | 0
wal_bytes | 2607518
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 249.174
-[ RECORD 1 ]--------+------------------
relname | pgbench_accounts
total_blks_read | 20513
total_blks_hit | 102219
total_blks_dirtied | 13973
total_blks_written | 13870
rel_blks_read | 14173
rel_blks_hit | 51664
pages_scanned | 16452
pages_removed | 16451
tuples_deleted | 999995
tuples_frozen | 0
recently_dead_tuples | 0
index_vacuum_count | 1
wal_records | 68136
wal_fpi | 2759
wal_bytes | 32422971
blk_read_time | 0
blk_write_time | 0
delay_time | 0
total_time | 773.4699999999999
[text/x-patch] v14-0001-Machinery-for-grabbing-an-extended-vacuu.patch (70.1K, 5-v14-0001-Machinery-for-grabbing-an-extended-vacuu.patch)
download | inline diff:
From fb8a110aa416e3314dabbf316df1897c5e244008 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 12 Dec 2024 11:17:01 +0300
Subject: [PATCH 1/4] Machinery for grabbing an extended vacuum
statistics on heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 146 +++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 51 +++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 46 +++-
src/backend/utils/adt/pgstatfuncs.c | 144 +++++++++++
src/backend/utils/error/elog.c | 13 +
src/backend/utils/misc/guc_tables.c | 9 +
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 81 ++++++-
src/include/utils/elog.h | 1 +
src/include/utils/pgstat_internal.h | 2 +-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 53 +++++
src/test/regress/expected/rules.out | 43 +++-
.../expected/vacuum_tables_statistics.out | 225 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 180 ++++++++++++++
22 files changed, 1081 insertions(+), 18 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f2ca9430581..987bd529dec 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -165,6 +165,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -239,6 +240,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -292,6 +305,106 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -324,7 +437,14 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -342,7 +462,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -359,6 +479,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -591,6 +712,26 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ if(pgstat_track_vacuum_statistics)
+ {
+ extVacReport.recently_dead_tuples = 0;
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ }
+
/*
* Report results to the cumulative stats system, too.
*
@@ -605,7 +746,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 79b79d5982e..ecfb0340328 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index da9a8fe99f2..f3cb5b5bf90 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -691,7 +691,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_vacuum_count(C.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(C.oid) AS autovacuum_count,
pg_stat_get_analyze_count(C.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1381,3 +1383,50 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.vm_new_frozen_pages AS vm_new_frozen_pages,
+ stats.vm_new_visible_pages AS vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ stats.missed_dead_pages AS missed_dead_pages,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.recently_dead_tuples AS recently_dead_tuples,
+ stats.missed_dead_tuples AS missed_dead_tuples,
+
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index bb639ef51fb..fa484dbe639 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -102,6 +102,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2419,6 +2422,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 67cba17a564..d7d9873c8ea 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1047,6 +1047,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index b4e357c8a42..bee1461814b 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -190,7 +190,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = true;
/* ----------
* state shared with pgstat_*.c
@@ -260,7 +260,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -890,7 +889,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -959,7 +957,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1075,7 +1073,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1126,7 +1124,7 @@ pgstat_prep_snapshot(void)
}
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index faba8b64d23..bbb095389be 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -208,7 +210,8 @@ pgstat_drop_relation(Relation rel)
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -232,6 +235,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -857,6 +862,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -980,3 +988,39 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
+ dst->vm_new_visible_pages += src->vm_new_visible_pages;
+ dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+ dst->missed_dead_pages += src->missed_dead_pages;
+ dst->missed_dead_tuples += src->missed_dead_tuples;
+
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index cdf37403e9d..c544f6bd73c 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_TIMESTAMPTZ(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2063,3 +2069,141 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 25
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 289059435a9..041fd3730a2 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 8cf1afbad20..bddfaba5b09 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1478,6 +1478,15 @@ struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
+ gettext_noop("Collects vacuum statistics for table relations."),
+ NULL
+ },
+ &pgstat_track_vacuum_statistics,
+ true,
+ NULL, NULL, NULL
+ },
{
{"track_wal_io_timing", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects timing statistics for WAL I/O activity."),
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0f22c217235..efa60fa3bc7 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12416,4 +12416,22 @@
proargtypes => 'int2',
prosrc => 'gist_stratnum_identity' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables return stats values',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index ebfeef2f460..a0c3253916a 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -166,6 +166,52 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* number of index vacuumings */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -208,6 +254,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -266,7 +322,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAF
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB0
typedef struct PgStat_ArchiverStats
{
@@ -405,6 +461,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -478,6 +536,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -645,7 +708,8 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
@@ -696,6 +760,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -713,7 +788,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
/*
* Functions in pgstat_replslot.c
*/
@@ -784,6 +858,7 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
/*
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index e54eca5b489..e752c0ce015 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 7338bc1e28e..d9328be48a7 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -595,7 +595,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
*/
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..87f7e40b4a6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..e93dd4f626c 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5893d89573d
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,53 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 3014d047fef..dd795d58dfc 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1804,7 +1804,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_vacuum_count(c.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(c.oid) AS autovacuum_count,
pg_stat_get_analyze_count(c.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2188,7 +2190,9 @@ pg_stat_sys_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2236,9 +2240,42 @@ pg_stat_user_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schema,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.missed_dead_pages,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.missed_dead_tuples,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..0c05a812dd1
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,225 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+ relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_scanned | pages_removed | vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | missed_dead_pages | tuples_deleted | tuples_frozen | recently_dead_tuples | missed_dead_tuples | index_vacuum_count | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+---------------+---------------------+----------------------+-----------------------------+-------------------+----------------+---------------+----------------------+--------------------+--------------------+-------------+---------+-----------+---------------+----------------+------------+------------
+ vestat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
+(1 row)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
+---------------------+----------------------+----------------------+-----------------------+-----------------------------
+ 0 | 910 | 0 | 0 | 455
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | f | f
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 1edd9e45ebb..d11f6b7ef4b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..8ad69108ca1
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,180 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+DROP TABLE vestat CASCADE;
--
2.34.1
[text/x-patch] v14-0002-Machinery-for-grabbing-an-extended-vacuu.patch (60.5K, 6-v14-0002-Machinery-for-grabbing-an-extended-vacuu.patch)
download | inline diff:
From a9d7ab3ae86a6ab21fdc0de9278ffaa78ed237fe Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 16 Dec 2024 12:06:17 +0300
Subject: [PATCH 2/4] Machinery for grabbing an extended vacuum
statistics on index relations.
They are gathered separatelly from table statistics.
As for tables, we gather vacuum shared buffers statistics for index relations like
value of total_blks_hit, total_blks_read, total_blks_dirtied, wal statistics, io time
during flushing buffer pages to disk, delay and total time.
Due to the fact taht such statistics are common as for tables, as for indexes we
set them in the union ExtVacReport structure. We only added some determination 'type'
field to highlight what kind belong to these statistics: PGSTAT_EXTVAC_TABLE or
PGSTAT_EXTVAC_INDEX. Generally, PGSTAT_EXTVAC_INVALID type leads to wrong code process.
Some statistics belong only one type of both tables or indexes. So, we added substructures
table and index inside ExtVacReport structure.
Therefore, we gather only for tables such statistics like number of scanned, removed pages,
their charecteristics according VM (all-visible and frozen). In addition, for tables we
gather number frozen, deleted and recently dead tuples and how many times vacuum processed
indexes for tables.
Controversally for indexes we gather number of deleted pages and deleted tuples only.
As for tables, deleted pages and deleted tuples reflect the overall performance of the vacuum
for the index relationship
Since vacuum clears tuple index references before clearing table tuples, we need to subtract
number of collected index vacuum statistics for general statistics taken into account
for the table and for the index, especially for
shared buffers: total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written;
wal statistics: wal_bytes, wal_fpi, wal_records; IO time: blk_read_time, blk_write_time;
total time and delay time. This is necessary to take into account vacuum statistics for tables
and indexes separately.
Due to the fact that the vacuum can produce workers to process indexes of the table and
workers store their wal and buffer statistic information separately from the place that
leader does we need to store an information about ParallelWorkerNumber that's why we
added the variable id_parallel_worker in the index's statistic structure. PVIndStats
stryucture store statistic information for every index whether it was processed by
leader or worker, so we are sure that id_parallel_worker will be updated correctly.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 165 ++++++++++++----
src/backend/catalog/system_views.sql | 32 +++
src/backend/commands/vacuumparallel.c | 59 +++++-
src/backend/utils/activity/pgstat.c | 4 +
src/backend/utils/activity/pgstat_relation.c | 50 +++--
src/backend/utils/adt/pgstatfuncs.c | 131 ++++++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 26 +++
src/include/pgstat.h | 59 ++++--
src/include/utils/pgstat_internal.h | 1 -
src/test/regress/expected/rules.out | 22 +++
.../expected/vacuum_index_statistics.out | 183 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 40 +++-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 151 +++++++++++++++
.../regress/sql/vacuum_tables_statistics.sql | 12 ++
17 files changed, 860 insertions(+), 87 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 987bd529dec..3f3ab1118cf 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -166,6 +166,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -240,19 +241,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-/*
- * Counters and usage data for extended stats tracking.
- */
-typedef struct LVExtStatCounters
-{
- TimestampTz starttime;
- WalUsage walusage;
- BufferUsage bufusage;
- double VacuumDelayTime;
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-} LVExtStatCounters;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -350,7 +338,8 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters *counters,
- ExtVacReport *report)
+ ExtVacReport *report, BufferUsage *worker_bufferusage,
+ WalUsage *worker_walusage)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -363,10 +352,16 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
/* Calculate diffs of global stat parameters on WAL and buffer usage. */
memset(&walusage, 0, sizeof(WalUsage));
- WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ if(worker_walusage == NULL)
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ else
+ WalUsageAccumDiff(&walusage, worker_walusage, &counters->walusage);
memset(&bufusage, 0, sizeof(BufferUsage));
- BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ if(worker_bufferusage == NULL)
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ else
+ BufferUsageAccumDiff(&bufusage, worker_bufferusage, &counters->bufusage);
endtime = GetCurrentTimestamp();
TimestampDifference(counters->starttime, endtime, &secs, &usecs);
@@ -406,6 +401,59 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ BufferUsage *buffusage, WalUsage *walusage)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if(buffusage != NULL)
+ memset(&(counters->common.bufusage), 0, sizeof(BufferUsage));
+
+ if(walusage != NULL)
+ memset(&(counters->common.walusage), 0, sizeof(WalUsage));
+
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report,
+ BufferUsage *buffusage, WalUsage *walusage)
+{
+ extvac_stats_end(rel, &counters->common, report,
+ buffusage, walusage);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -440,11 +488,6 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
LVExtStatCounters extVacCounters;
ExtVacReport extVacReport;
char **indnames = NULL;
- ExtVacReport allzero;
-
- /* Initialize vacuum statistics */
- memset(&allzero, 0, sizeof(ExtVacReport));
- extVacReport = allzero;
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -711,25 +754,35 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
new_rel_allvisible, vacrel->nindexes > 0,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
-
- /* Make generic extended vacuum stats report */
- extvac_stats_end(rel, &extVacCounters, &extVacReport);
+ extvac_stats_end(rel, &extVacCounters, &extVacReport, NULL, NULL);
if(pgstat_track_vacuum_statistics)
{
- extVacReport.recently_dead_tuples = 0;
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
- extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
- extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.type = PGSTAT_EXTVAC_TABLE;
+ extVacReport.table.pages_scanned = vacrel->scanned_pages;
+ extVacReport.table.pages_removed = vacrel->removed_pages;
+ extVacReport.table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
- extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.table.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.table.index_vacuum_count = vacrel->num_index_scans;
+
+ }
+ else
+ {
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics with 0 values to prevent
+ * adding garbage values in memory
+ */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
}
/*
@@ -2504,6 +2557,7 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->vm_new_frozen_pages++;
}
PageSetAllVisible(page);
@@ -2671,6 +2725,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters, NULL, NULL);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2689,6 +2747,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2697,6 +2756,15 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport, NULL, NULL);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2721,6 +2789,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters, NULL, NULL);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2740,12 +2812,22 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport, NULL, NULL);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3321,6 +3403,8 @@ update_relstats_all_indexes(LVRelState *vacrel)
Relation *indrels = vacrel->indrels;
int nindexes = vacrel->nindexes;
IndexBulkDeleteResult **indstats = vacrel->indstats;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
Assert(vacrel->do_index_cleanup);
@@ -3332,6 +3416,8 @@ update_relstats_all_indexes(LVRelState *vacrel)
if (istat == NULL || istat->estimated_count)
continue;
+ extvac_stats_start_idx(indrel, istat, &extVacCounters, NULL, NULL);
+
/* Update index statistics */
vac_update_relstats(indrel,
istat->num_pages,
@@ -3341,6 +3427,15 @@ update_relstats_all_indexes(LVRelState *vacrel)
InvalidTransactionId,
InvalidMultiXactId,
NULL, NULL, false);
+
+ if(pgstat_track_vacuum_statistics)
+ {
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport, NULL, NULL);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+ }
}
}
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index f3cb5b5bf90..30e5c0a7a44 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1430,3 +1430,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index d7d9873c8ea..e966da4de7f 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -154,6 +154,10 @@ typedef struct PVIndStats
*/
bool istat_updated; /* are the stats updated? */
IndexBulkDeleteResult istat;
+
+ LVExtStatCountersIdx counters;
+ ExtVacReport idx_report;
+ int id_parallel_worker; /* detect index was processed by postmster or worker */
} PVIndStats;
/*
@@ -654,6 +658,8 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
{
PVIndStats *indstats = &(pvs->indstats[i]);
+ indstats->id_parallel_worker = -2;
+
Assert(indstats->status == PARALLEL_INDVAC_STATUS_INITIAL);
indstats->status = new_status;
indstats->parallel_workers_can_process =
@@ -661,6 +667,12 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
parallel_vacuum_index_is_parallel_safe(pvs->indrels[i],
num_index_scans,
vacuum));
+
+ /* Sava buffer and wal statistics before vacuuming to track them
+ * for the leader.
+ */
+ extvac_stats_start_idx(pvs->indrels[i], &(indstats->istat),
+ &(indstats->counters),NULL, NULL);
}
/* Reset the parallel index processing and progress counters */
@@ -727,19 +739,10 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
*/
parallel_vacuum_process_safe_indexes(pvs);
- /*
- * Next, accumulate buffer and WAL usage. (This must wait for the workers
- * to finish, or we might get incomplete data.)
- */
if (nworkers > 0)
- {
/* Wait for all vacuum workers to finish */
WaitForParallelWorkersToFinish(pvs->pcxt);
- for (int i = 0; i < pvs->pcxt->nworkers_launched; i++)
- InstrAccumParallelQuery(&pvs->buffer_usage[i], &pvs->wal_usage[i]);
- }
-
/*
* Reset all index status back to initial (while checking that we have
* vacuumed all indexes).
@@ -752,9 +755,44 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
elog(ERROR, "parallel index vacuum on index \"%s\" is not completed",
RelationGetRelationName(pvs->indrels[i]));
+ /* If an index was processed by worker we need to gather wal and
+ * buffer statistics from pvs->buffer_usage and pvs->wal_usage,
+ * otherwice for leader they can be detected through substract
+ * of global statistics of pgWalUsage and pgBufferUsage.
+ */
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* We expect that all indexes have updated it */
+ Assert(indstats->id_parallel_worker != -2);
+
+ if(indstats->id_parallel_worker == -1)
+ extvac_stats_end_idx(pvs->indrels[i], &(indstats->istat), &(indstats->counters),
+ &(indstats->idx_report), NULL, NULL);
+ else
+ {
+ /* We need to reset Buffer and Wal usage statistics */
+ memset(&(indstats->counters.common.bufusage), 0, sizeof(BufferUsage));
+ memset(&(indstats->counters.common.walusage), 0, sizeof(WalUsage));
+ extvac_stats_end_idx(pvs->indrels[i], &(indstats->istat), &(indstats->counters),
+ &(indstats->idx_report), &pvs->buffer_usage[indstats->id_parallel_worker], &pvs->wal_usage[indstats->id_parallel_worker]);
+ }
+
+ pgstat_report_vacuum(RelationGetRelid(pvs->indrels[i]),
+ pvs->indrels[i]->rd_rel->relisshared,
+ 0, 0, &(indstats->idx_report));
+ }
+
indstats->status = PARALLEL_INDVAC_STATUS_INITIAL;
}
+ /*
+ * Next, accumulate buffer and WAL usage. (This must wait for the workers
+ * to finish, or we might get incomplete data.)
+ */
+ if (nworkers > 0)
+ for (int i = 0; i < pvs->pcxt->nworkers_launched; i++)
+ InstrAccumParallelQuery(&pvs->buffer_usage[i], &pvs->wal_usage[i]);
+
/*
* Carry the shared balance value to heap scan and disable shared costing
*/
@@ -925,6 +963,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
pfree(istat_res);
}
+ /* Like ParallelWorkerNumber can be -1 for leader and more 0 for workers */
+ indstats->id_parallel_worker = ParallelWorkerNumber;
+
/*
* Update the status to completed. No need to lock here since each worker
* touches different indexes.
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index bee1461814b..85c544cf956 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1169,6 +1169,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bbb095389be..cc6b46eb7db 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
@@ -989,10 +987,13 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
}
}
-static void
+void
pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
bool accumulate_reltype_specific_info)
{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
dst->total_blks_read += src->total_blks_read;
dst->total_blks_hit += src->total_blks_hit;
dst->total_blks_dirtied += src->total_blks_dirtied;
@@ -1008,19 +1009,34 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
- dst->vm_new_visible_pages += src->vm_new_visible_pages;
- dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
- dst->missed_dead_pages += src->missed_dead_pages;
- dst->missed_dead_tuples += src->missed_dead_tuples;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->tuples_deleted += src->tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c544f6bd73c..acdf16a98d5 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2175,17 +2175,18 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2204,6 +2205,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index bddfaba5b09..430d73cc3df 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1480,7 +1480,7 @@ struct config_bool ConfigureNamesBool[] =
},
{
{"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
- gettext_noop("Collects vacuum statistics for table relations."),
+ gettext_noop("Collects vacuum statistics for relations."),
NULL
},
&pgstat_track_vacuum_statistics,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index efa60fa3bc7..6e2953e9dd5 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12434,4 +12434,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 07b28b15d9f..93182e0a8a7 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -288,6 +289,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -386,4 +407,9 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, BufferUsage *buffusage, WalUsage *walusage);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report,
+ BufferUsage *buffusage, WalUsage *walusage);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index a0c3253916a..9ff41517e4f 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -166,11 +166,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -199,17 +207,43 @@ typedef struct ExtVacReport
double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
double total_time; /* total time of a vacuum operation, in msec */
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
- int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
- int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
- int64 missed_dead_pages; /* pages with missed dead tuples */
int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
- int64 index_vacuum_count; /* number of index vacuumings */
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -702,7 +736,8 @@ extern PgStat_FunctionCounts *find_funcstat_entry(Oid func_id);
extern void pgstat_create_relation(Relation rel);
extern void pgstat_drop_relation(Relation rel);
extern void pgstat_copy_relation_stats(Relation dst, Relation src);
-
+extern void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
extern void pgstat_init_relation(Relation rel);
extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index d9328be48a7..8b296c61cf7 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -595,7 +595,6 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
*/
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index dd795d58dfc..2becc7f3885 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2245,6 +2245,28 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT ns.nspname AS schema,
rel.relname,
stats.relid,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..b840a6ed4fe
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+ relid | schema | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+--------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+(0 rows)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index 0c05a812dd1..119c7abea5f 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -181,17 +181,39 @@ WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
vestat | t | t | f | t | t
(1 row)
+DROP TABLE vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
-- must be empty
SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
---------------------+----------------------+----------------------+-----------------------+-----------------------------
- 0 | 910 | 0 | 0 | 455
+ 0 | 0 | 0 | 0 | 0
(1 row)
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
-- backend defreezed pages
SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
@@ -204,16 +226,28 @@ SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_froz
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
UPDATE vestat SET x = x + 1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
---------------------+----------------------+-----------------------------+----------------------+-----------------------
- f | t | f | f | f
+ f | t | t | t | t
(1 row)
SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
-- vacuum freezed pages
SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
@@ -222,4 +256,6 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
t | t | t | t | t
(1 row)
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index d11f6b7ef4b..977a87a5b1f 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..ae146e1d23f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,151 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
index 8ad69108ca1..dfd7af70027 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -148,14 +148,22 @@ SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_dele
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+DROP TABLE vestat;
+SELECT pg_stat_force_next_flush();
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
+SELECT pg_stat_force_next_flush();
-- must be empty
SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
-- backend defreezed pages
SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
@@ -165,6 +173,7 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
UPDATE vestat SET x = x + 1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
@@ -172,9 +181,12 @@ SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_fro
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
-- vacuum freezed pages
SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
--
2.34.1
[text/x-patch] v14-0003-Machinery-for-grabbing-an-extended-vacuu.patch (21.5K, 7-v14-0003-Machinery-for-grabbing-an-extended-vacuu.patch)
download | inline diff:
From f9a080cf184613dcf3db8a1333cd5fbddddc08a6 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 16 Dec 2024 14:11:30 +0300
Subject: [PATCH 3/4] Machinery for grabbing an extended vacuum
statistics on databases.
Database vacuum statistics information is the collected general
vacuum statistics indexes and tables owned by the databases, which
they belong to.
So, buffer, wal, statistics of I/O time of read and writen blocks
statistics will be observed because they are collected for both
tables, indexes. In addition, we show the number of errors caught
during operation of the vacuum only for the error level.
---
src/backend/access/heap/vacuumlazy.c | 15 +++
src/backend/catalog/system_views.sql | 26 ++++-
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 43 +++++++++
src/backend/utils/adt/pgstatfuncs.c | 95 +++++++++++++++++++
src/include/catalog/pg_proc.dat | 15 ++-
src/include/pgstat.h | 3 +
src/test/regress/expected/rules.out | 16 ++++
...ut => vacuum_tables_and_db_statistics.out} | 69 +++++++++++++-
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 60 +++++++++++-
11 files changed, 338 insertions(+), 7 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (84%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (84%)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3f3ab1118cf..5279e22a204 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -3453,6 +3453,9 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3468,6 +3471,9 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3483,16 +3489,25 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 30e5c0a7a44..b053d8edc33 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1461,4 +1461,28 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.errors AS errors
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 7757d2ace74..840d848a752 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -449,6 +449,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index cc6b46eb7db..37baa7112d9 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -203,6 +203,38 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.type = m_type;
+ pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.errors++;
+ dbentry->vacuum_ext.type = m_type;
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
@@ -214,6 +246,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -267,6 +300,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index acdf16a98d5..d9595b69492 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2315,6 +2315,101 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 13
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6e2953e9dd5..016dacb4716 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12417,7 +12417,7 @@
prosrc => 'gist_stratnum_identity' },
{ oid => '8001',
- descr => 'pg_stat_get_vacuum_tables return stats values',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
@@ -12435,12 +12435,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 9ff41517e4f..a6e2bb4d475 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -209,6 +209,8 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
+ int32 errors;
+
ExtVacReportType type; /* heap, index, etc. */
/* ----------
@@ -748,6 +750,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2becc7f3885..9d84fba378c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2245,6 +2245,22 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, errors);
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 84%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index 119c7abea5f..5efe8998abe 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,7 +6,6 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
--------------
@@ -43,6 +42,11 @@ SHOW track_vacuum_statistics; -- must be on
on
(1 row)
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -259,3 +263,66 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
RESET vacuum_freeze_min_age;
RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+RESET track_vacuum_statistics;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 977a87a5b1f..19c76b96830 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 84%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index dfd7af70027..d0e4a2014c6 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,7 +7,6 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
\set sample_size 10000
@@ -38,6 +37,13 @@ DROP TABLE vestat CASCADE;
SHOW track_vacuum_statistics; -- must be on
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -190,3 +196,55 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
RESET vacuum_freeze_min_age;
RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+RESET track_vacuum_statistics;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
--
2.34.1
[text/x-patch] v14-0004-Add-documentation-about-the-system-views.patch (24.0K, 8-v14-0004-Add-documentation-about-the-system-views.patch)
download | inline diff:
From e1c78a3f3aa45ad5f1fc5466f83ce82eb4ad4bfc Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 4/4] Add documentation about the system views
that are used in the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 737 +++++++++++++++++++++++++++++++++
1 file changed, 737 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index a586156614d..60ce035e60f 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5066,4 +5066,741 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-get-vacuum-database">
+ <title><structname>pg_stat_get_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-database">
+ <primary>pg_stat_get_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-get-vacuum-indexes">
+ <title><structname>pg_stat_get_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-indexes">
+ <primary>pg_stat_get_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-get-vacuum-tables">
+ <title><structname>pg_stat_get_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-get-vacuum-tables">
+ <primary>pg_stat_get_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_get_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_get_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.34.1
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2024-12-19 10:40 Alena Rybakina <[email protected]>
parent: Ilia Evdokimov <[email protected]>
1 sibling, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2024-12-19 10:40 UTC (permalink / raw)
To: Ilia Evdokimov <[email protected]>; +Cc: Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
Hi!
On 02.12.2024 17:46, Ilia Evdokimov wrote:
> In my opinion, the patches are semantically correct. However, not all
> dead code has been removed - I'm referring to
> pgstat_update_snapshot(). Also, the tests need to be fixed.
>
>
I fixed it [0]. Thank you!
[0]
https://www.postgresql.org/message-id/86f76aa5-1ab5-4e2e-9b15-405051852a2a%40postgrespro.ru
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-02 20:12 Sami Imseih <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 2 replies; 46+ messages in thread
From: Sami Imseih @ 2025-01-02 20:12 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
Hi,
Thanks for the work you have done here. Exposing cumulative
metrics at this level of detail for vacuum is surely useful to find
vacuum bottlenecks and to determine the effectiveness of
vacuum tuning.
I am yet to look very closely, but I think some additional columns that
will be useful is the number of failsafe autovacuums occurred. Also
the counter for number of index_cleanup skipped, truncate phase
skipped and toast vacuuming skipped ( the latter will only be relevant
for the main relation ).
I also wonder if if makes sense to break down timing by phase. I surely
would like to know how much of my vacuum time was spent in index
cleanup vs heap scan, etc.
A nit: I noticed in v14, the column is "schema". It should be "schemaname"
for consistency.
Also, instead of pg_stat_vacuum_tables, what about pg_stat_vacuum?
Now, I became aware of this discussion after starting a new thread
to track total time spent in vacuum/analyze in pg_stat_all_tables [1].
But this begs the question of what should be done with the current
counters in pg_stat_all_tables? I see it mentioned above that (auto)vacuum_count
should be added to this new view, but it's also already in pg_stat_all_tables.
I don't think we should be duplicating the same columns across views.
I think total_time should be removed from your current patch and added
as is being suggested in [1]. This way high level metrics such as counts
and total time spent remain in pg_stat_all_tables, while the new view
you are proposing will contain more details. I don't think we will have
consistency issues between the views because a reset using pg_stat_reset()
will act on all the stats and pg_stat_reset_single_table_counters() will act on
all the stats related to that table. There should be no way to reset the vacuum
stats independently, AFAICT.
Alternatively, we can remove the vacuum related stats from pg_stat_all_tables,
but that will break monitoring tools and will leave us with the (auto)analyze
metrics alone in pg_stat_all_tables. This sounds very ugly.
What do you think?
Regards,
Sami Imseih
Amazon Web Services (AWS)
[1] https://commitfest.postgresql.org/52/5485/
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-02 21:14 Jim Nasby <[email protected]>
parent: Sami Imseih <[email protected]>
1 sibling, 2 replies; 46+ messages in thread
From: Jim Nasby @ 2025-01-02 21:14 UTC (permalink / raw)
To: Sami Imseih <[email protected]>; +Cc: Alena Rybakina <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
> On Jan 2, 2025, at 2:12 PM, Sami Imseih <[email protected]> wrote:
>
> Alternatively, we can remove the vacuum related stats from pg_stat_all_tables,
> but that will break monitoring tools and will leave us with the (auto)analyze
> metrics alone in pg_stat_all_tables. This sounds very ugly.
While backwards compatibility is important, there’s definitely precedent for changing what shows up in the catalog. IMHO it’s better to bite the bullet and move those fields instead of having vacuum stats spread across two different views.
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-02 22:33 Sami Imseih <[email protected]>
parent: Jim Nasby <[email protected]>
1 sibling, 1 reply; 46+ messages in thread
From: Sami Imseih @ 2025-01-02 22:33 UTC (permalink / raw)
To: Jim Nasby <[email protected]>; +Cc: Alena Rybakina <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
> While backwards compatibility is important, there’s definitely precedent for changing
> what shows up in the catalog. IMHO it’s better to bite the bullet and move those fields
> instead of having vacuum stats spread across two different views.
Correct, the most recent one that I could think of is pg_stat_checkpointer,
which pulled the checkpoint related columns from pg_stat_bgwriter.
In that case though, these are distinct background processes and
it's a clear distinction.
In this case, I am not so sure about this, particularly because
we will then have the autoanalyze and autovacuum fields in different
views, which could be more confusing to users than saying pg_stat_all_tables
has high level metrics about vacuum and analyze and for more details on
vacuum, refer to pg_stat_vacuum_tables ( or whatever name we settle on ).
Regards,
Sami
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-03 14:15 Greg Sabino Mullane <[email protected]>
parent: Jim Nasby <[email protected]>
1 sibling, 0 replies; 46+ messages in thread
From: Greg Sabino Mullane @ 2025-01-03 14:15 UTC (permalink / raw)
To: Jim Nasby <[email protected]>; +Cc: Sami Imseih <[email protected]>; Alena Rybakina <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
>
> While backwards compatibility is important, there’s definitely precedent
> for changing what shows up in the catalog. IMHO it’s better to bite the
> bullet and move those fields instead of having vacuum stats spread across
> two different views.
>
-1. That's a huge change, and pg_stat_all_tables is used way, way more than
things like pg_stat_bgwriter.
Cheers,
Greg
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-03 19:08 Jim Nasby <[email protected]>
parent: Sami Imseih <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Jim Nasby @ 2025-01-03 19:08 UTC (permalink / raw)
To: Sami Imseih <[email protected]>; +Cc: Alena Rybakina <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
On Jan 2, 2025, at 4:33 PM, Sami Imseih <[email protected]> wrote:
>
>> While backwards compatibility is important, there’s definitely precedent for changing
>> what shows up in the catalog. IMHO it’s better to bite the bullet and move those fields
>> instead of having vacuum stats spread across two different views.
>
> Correct, the most recent one that I could think of is pg_stat_checkpointer,
> which pulled the checkpoint related columns from pg_stat_bgwriter.
> In that case though, these are distinct background processes and
> it's a clear distinction.
>
> In this case, I am not so sure about this, particularly because
> we will then have the autoanalyze and autovacuum fields in different
> views, which could be more confusing to users than saying pg_stat_all_tables
> has high level metrics about vacuum and analyze and for more details on
> vacuum, refer to pg_stat_vacuum_tables ( or whatever name we settle on ).
I guess one question is how realistic it is to try and put everything about (auto)vacuum in a single view. Given the complexity, the answer to that might just be “no”. In that case leaving existing fields in pg_stat_all_tables is a lot more reasonable.
Related to this… it’d be nice if we had a view that gave insight to users about auto vacuum scheduling. I know there’s one floating around the internet, but given the number of systems I’ve seen where autovac can’t keep up it’d be good to raise user awareness.
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-03 19:46 Sami Imseih <[email protected]>
parent: Jim Nasby <[email protected]>
0 siblings, 0 replies; 46+ messages in thread
From: Sami Imseih @ 2025-01-03 19:46 UTC (permalink / raw)
To: Jim Nasby <[email protected]>; +Cc: Alena Rybakina <[email protected]>; Ilia Evdokimov <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
> I guess one question is how realistic it is to try and put everything about (auto)vacuum in a single view.
> Given the complexity, the answer to that might just be “no”. In that case leaving existing fields in pg_stat_all_tables
> is a lot more reasonable.
Agree. I also think the total_time should be in pg_stat_all_tables.
total_time is a high level metric that along with vacuum_count
can calculate average run time of vacuums on a specific table.
Everything else in the new view are more granular details.
Regards,
Sami
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-04 20:37 Alena Rybakina <[email protected]>
parent: Sami Imseih <[email protected]>
1 sibling, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2025-01-04 20:37 UTC (permalink / raw)
To: Sami Imseih <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
Hi, thank you for your attention to this patch.
On 02.01.2025 23:12, Sami Imseih wrote:
> Hi,
>
> Thanks for the work you have done here. Exposing cumulative
> metrics at this level of detail for vacuum is surely useful to find
> vacuum bottlenecks and to determine the effectiveness of
> vacuum tuning.
Yes, we hope that this will help provide more detailed information about
the current efficiency of the vacuum and also suggest how to best
configure it for the relationship.
> I am yet to look very closely, but I think some additional columns that
> will be useful is the number of failsafe autovacuums occurred.
Do you mean when the autovacuum started to prevent workaround?
> Also
> the counter for number of index_cleanup skipped, truncate phase
> skipped and toast vacuuming skipped ( the latter will only be relevant
> for the main relation ).
I can add, but concerns have already been expressed about the large
amount of vacuum statistics and, as a consequence, this leads to the
allocation of additional memory (3 times).
Of course, now we are saved by the guc I added for statistics... I
understand that this information can better show the efficiency of the
vacuum, but how does it help in setting it up for heap relations?
regarding the skipped truncate phase, the statistics are already
collected in vacrel->nonempty_pages, it's easy to put them outside. I
think the current statistics only show the number of deleted tuples and
pages (both deleted and those visited by vacuum during tuple deletion),
so the opposite view won't hurt.
index_cleanup skipped can be obtained based on information from a small
number of vacuum buffer statistics and the number of pages of indexes
that belong to heap relations. I think you can notice the behavior
through current statistics: if the index's buffer values have increased
very slightly, then the vacuum does not go there probably because of the
impossibility of taking a clean-up lock on the index. The same
information can be obtained based on the number of missed_tuples in heap
relations. I wrote earlier how these values are related.
toast vacuuming skipped to be honest I haven't found a place where
vacuum skips it in the code yet, so I can't say anything about them yet.
> I also wonder if if makes sense to break down timing by phase. I surely
> would like to know how much of my vacuum time was spent in index
> cleanup vs heap scan, etc.
At the moment, this information has already been added to the statistics
as a total time for heap relations and their indexes.
>
> A nit: I noticed in v14, the column is "schema". It should be "schemaname"
> for consistency.
Thank you, I'll fix it in the next version of the patch.
> Also, instead of pg_stat_vacuum_tables, what about pg_stat_vacuum?
>
> Now, I became aware of this discussion after starting a new thread
> to track total time spent in vacuum/analyze in pg_stat_all_tables [1].
> But this begs the question of what should be done with the current
> counters in pg_stat_all_tables? I see it mentioned above that (auto)vacuum_count
> should be added to this new view, but it's also already in pg_stat_all_tables.
> I don't think we should be duplicating the same columns across views.
>
> Alternatively, we can remove the vacuum related stats from pg_stat_all_tables,
> but that will break monitoring tools and will leave us with the (auto)analyze
> metrics alone in pg_stat_all_tables. This sounds very ugly.
>
> What do you think?
>
> Regards,
>
> Sami Imseih
> Amazon Web Services (AWS)
>
> [1]https://commitfest.postgresql.org/52/5485/
I don't think they interfere with my more detailed views of how the
vacuum works. I don't think there's anything worth removing.
>
> I think total_time should be removed from your current patch and added
> as is being suggested in [1]. This way high level metrics such as counts
> and total time spent remain in pg_stat_all_tables, while the new view
> you are proposing will contain more details. I don't think we will have
> consistency issues between the views because a reset using pg_stat_reset()
> will act on all the stats and pg_stat_reset_single_table_counters() will act on
> all the stats related to that table. There should be no way to reset the vacuum
> stats independently, AFAICT.
I think it is not quite correct to do so.
Firstly, the total time of vacuum operation does not give you a complete
idea of when vacuum did not work delay time. I have seen many reports
where vacuum spends very little time on cleaning relations and most of
the time just sleeping.
Secondly, where to put the total time of vacuum for indexes and
databases? It would be incorrect not to take them into account at all.
What if we remove the total time from the heap statistics and add it to
pg_stat_tables and only leave the vacuum statistics total time of vacuum
operation of indexes and databases? It seems strange to me that they
will have to be viewed from different views.
I think it is necessary to look at the total time for tables into
perspective of how much time vacuum spent in total on processing
indexes, since indexes can be bloated, for example. I think it is better
to leave these statistics here.
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-06 02:00 Sami Imseih <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 2 replies; 46+ messages in thread
From: Sami Imseih @ 2025-01-06 02:00 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
> I am yet to look very closely, but I think some additional columns that
> will be useful is the number of failsafe autovacuums occurred.
>
> Do you mean when the autovacuum started to prevent workaround?
>
Specifically vacuum_failsafe_age [1] when autovacuum automatically
performs a vacuum without index cleanup, without truncate, bypassing
the vacuum ring buffer and disabling the cost limits. The purpose of this
is a last ditch effort to avoid wraparound and is triggered at 1.6 billion
transactions by default. When this state occurs, there is a single log
written for every table that is vacuumed with these options [2], and
my thoughts is to also track in the view as the use of these options
will overtime make the indexes bloat over time and less space is
given back to the OS due to skipped truncations. For most workloads,
this should not be common, but I am thinking of the extreme cases
or if someone potentially misconfigured the vacuum_failsafe_age.
As I thought about this more, failsafe autovacuum could be tracked on
the database level, pg_stat_database, since this guc can't be set
on a relation level.
> Also
> the counter for number of index_cleanup skipped, truncate phase
> skipped and toast vacuuming skipped ( the latter will only be relevant
> for the main relation ).
>
> I can add, but concerns have already been expressed about the large amount of
> vacuum statistics and, as a consequence, this leads
> to the allocation of additional memory (3 times).
> Of course, now we are saved by the guc I added for statistics...
> I understand that this information can better show the efficiency of the vacuum,
> but how does it help in setting it up for heap relations?
An administrator will find this information to be useful especially
if for some reason most vacuums are being run with these
options being off either via a manual vacuum or someone
turning off index_cleanup in the tables storage parameter.
postgres=# alter table t set (vacuum_index_cleanup = off,
vacuum_truncate = off );
ALTER TABLE
> regarding the skipped truncate phase, the statistics are already collected in vacrel->nonempty_pages,
> it's easy to put them outside. I think the current statistics only show the number of deleted tuples and pages
> (both deleted and those visited by vacuum during tuple deletion), so the opposite view won't hurt.
Can you clarify what you mean by "so the opposite view won't hurt." ?
> index_cleanup skipped can be obtained based on information from a small number of
> vacuum buffer statistics and the number of pages of indexes that belong to heap relations.
> I think you can notice the behavior through current statistics:
I don't think there is a view that provides cumulative vacuum buffer
stats currently.
pg_stat_io could be helpful for this purpose, but that is a cluster
wide view. As it
stands now, I think it's quite difficult for a user to determine for a
fact if indexes or
truncate is being skipped
> Secondly, where to put the total time of vacuum for indexes and databases?
> It would be incorrect not to take them into account at all. What if we remove the total time from
> the heap statistics and add it to pg_stat_tables and only leave the vacuum statistics total time of
> vacuum operation of indexes and databases?
> It seems strange to me that they will have to be viewed from different views.
>
> I think it is necessary to look at the total time for tables into perspective of how much
> time vacuum spent in total on processing indexes, since indexes can be bloated, for example.
> I think it is better to leave these statistics here.
You make valid points. I now think because track_vacuum_statistics is
optional, we should track total_time in 2 places. First place in the new
view being proposed here and the second place is in pg_stat_all_tables
as being proposed here [3]. This way if track_vacuum_statistics is off, the
total_time of vacuum could still be tracked by pg_stat_all_tables.
By the way, the current patch does not track materialized view,
but it should as materialized views can also be vacuumed.
Regards,
Sami
[1] https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-VACUUM-FAILSAFE-AGE
[2] https://github.com/postgres/postgres/blob/master/src/backend/access/heap/vacuumlazy.c#L2437-L2444
[3] https://commitfest.postgresql.org/52/5485/
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-10 12:04 Alena Rybakina <[email protected]>
parent: Sami Imseih <[email protected]>
1 sibling, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2025-01-10 12:04 UTC (permalink / raw)
To: Sami Imseih <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Alexander Korotkov <[email protected]>; Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
Hi, I have updated the patch. Fix minor mistakes in the document, added
the wraparound_failsafe_count statistics - it accounts the number of
times when the vacuum operates heap relation to prevent workaround
problem, fixed "shemaname".
On 06.01.2025 05:00, Sami Imseih wrote:
>> I am yet to look very closely, but I think some additional columns that
>> will be useful is the number of failsafe autovacuums occurred.
>>
>> Do you mean when the autovacuum started to prevent workaround?
>>
> Specifically vacuum_failsafe_age [1] when autovacuum automatically
> performs a vacuum without index cleanup, without truncate, bypassing
> the vacuum ring buffer and disabling the cost limits. The purpose of this
> is a last ditch effort to avoid wraparound and is triggered at 1.6 billion
> transactions by default. When this state occurs, there is a single log
> written for every table that is vacuumed with these options [2], and
> my thoughts is to also track in the view as the use of these options
> will overtime make the indexes bloat over time and less space is
> given back to the OS due to skipped truncations. For most workloads,
> this should not be common, but I am thinking of the extreme cases
> or if someone potentially misconfigured the vacuum_failsafe_age.
>
> As I thought about this more, failsafe autovacuum could be tracked on
> the database level, pg_stat_database, since this guc can't be set
> on a relation level.
I thought again about adding a statistic to account for skipping
truncation or index scans. In my opinion, we have statistics like
removed_pages on the heap relations. They are "the count number pages
removed by relation truncation". So if truncation was disabled on the
heap relation, their count will not increase.
As for skipped_indexes, we added an index_vacuum_count statistic that
counts the number of indexes on the heap relation that were vacuumed
during the vacuum procedure. If their count does not increase, then
vacuum will likely skip them.
>> Also
>> the counter for number of index_cleanup skipped, truncate phase
>> skipped and toast vacuuming skipped ( the latter will only be relevant
>> for the main relation ).
>>
>> I can add, but concerns have already been expressed about the large amount of
>> vacuum statistics and, as a consequence, this leads
>> to the allocation of additional memory (3 times).
>> Of course, now we are saved by the guc I added for statistics...
>> I understand that this information can better show the efficiency of the vacuum,
>> but how does it help in setting it up for heap relations?
> An administrator will find this information to be useful especially
> if for some reason most vacuums are being run with these
> options being off either via a manual vacuum or someone
> turning off index_cleanup in the tables storage parameter.
>
> postgres=# alter table t set (vacuum_index_cleanup = off,
> vacuum_truncate = off );
> ALTER TABLE
You can take these parameters into account when analyzing vacuum
statistics, right? Display them side by side.
>> regarding the skipped truncate phase, the statistics are already collected in vacrel->nonempty_pages,
>> it's easy to put them outside. I think the current statistics only show the number of deleted tuples and pages
>> (both deleted and those visited by vacuum during tuple deletion), so the opposite view won't hurt.
> Can you clarify what you mean by "so the opposite view won't hurt." ?
I meant that it wouldn't be excessive, but at the moment I think
otherwise. We already have removed_pages and it will be enough.
>> index_cleanup skipped can be obtained based on information from a small number of
>> vacuum buffer statistics and the number of pages of indexes that belong to heap relations.
>> I think you can notice the behavior through current statistics:
> I don't think there is a view that provides cumulative vacuum buffer
> stats currently.
We show it now in the views for heap relations, index relations,
databases (pg_stat_vacuum_tables, pg_stat_vacuum_indexes,
pg_stat_vacuum_databases) or you meant something else?
> pg_stat_io could be helpful for this purpose, but that is a cluster
> wide view. As it
> stands now, I think it's quite difficult for a user to determine for a
> fact if indexes or
> truncate is being skipped
I think so, it is difficult to get a clear picture of what is happening
by analyzing only this information.
We collect other statistics on vacuumed relation pages that can help
give a full picture: the number of pages missed due to failure to get a
clean-up lock on an index (missed_tuples), the number of vacuumed tuples
(tuples_deleted), and recently deleted tuples (recently_dead_tuples). I
think that's enough.
>> Secondly, where to put the total time of vacuum for indexes and databases?
>> It would be incorrect not to take them into account at all. What if we remove the total time from
>> the heap statistics and add it to pg_stat_tables and only leave the vacuum statistics total time of
>> vacuum operation of indexes and databases?
>> It seems strange to me that they will have to be viewed from different views.
>>
>> I think it is necessary to look at the total time for tables into perspective of how much
>> time vacuum spent in total on processing indexes, since indexes can be bloated, for example.
>> I think it is better to leave these statistics here.
> You make valid points. I now think because track_vacuum_statistics is
> optional, we should track total_time in 2 places. First place in the new
> view being proposed here and the second place is in pg_stat_all_tables
> as being proposed here [3]. This way if track_vacuum_statistics is off, the
> total_time of vacuum could still be tracked by pg_stat_all_tables.
>
> By the way, the current patch does not track materialized view,
> but it should as materialized views can also be vacuumed.
>
> Regards,
>
> Sami
>
> [1]https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-VACUUM-FAILSAFE-AGE
> [2]https://github.com/postgres/postgres/blob/master/src/backend/access/heap/vacuumlazy.c#L2437-L2444
> [3]https://commitfest.postgresql.org/52/5485/
>
>
I don't agree with this.
Firstly, the hook is enabled by default, that is, it must be specially
disabled so that the vacuum statistics are not collected.
Secondly, it will cause confusion. First, the hook was disabled and
statistics were collected in one place - pg_stat_all_tables, and then it
was enabled and the user notices that the statistics there stopped
accumulating,
he is in a panic, "suddenly the vacuum does not work, what to do?". The
second point here bothers me, how to take into account this statistics
with the current detailed vacuum statistics? After all, adding these
values is wrong -
they do not show the correct statistics regarding the same pages
processed by vacuum, ignoring it later means that they will be
redundant. I think it is better to save it here, since this will save us
from possible confusion.
Secondly, it will immediately show other important parameters regarding
this statistics - how long the vacuum was sleep (delay_time in my
patches), how much time the vacuum spent on processing indexes during
its processing.
Without this information, this assessment will not be voluminous and
indicative enough.
--
Regards,
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] v15-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (71.5K, 3-v15-0001-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From 0cd8502efe988407c05edd8a8bbc813740696bf1 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Fri, 10 Jan 2025 09:57:13 +0300
Subject: [PATCH 1/4] Machinery for grabbing an extended vacuum statistics on
heap relations.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 150 +++++++++++-
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 52 +++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 47 +++-
src/backend/utils/adt/pgstatfuncs.c | 147 ++++++++++++
src/backend/utils/error/elog.c | 13 +
src/backend/utils/misc/guc_tables.c | 9 +
src/include/catalog/pg_proc.dat | 18 ++
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 82 ++++++-
src/include/utils/elog.h | 1 +
src/include/utils/pgstat_internal.h | 2 +-
.../vacuum-extending-in-repetable-read.out | 53 +++++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 53 +++++
src/test/regress/expected/rules.out | 43 +++-
.../expected/vacuum_tables_statistics.out | 225 ++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
.../regress/sql/vacuum_tables_statistics.sql | 180 ++++++++++++++
22 files changed, 1091 insertions(+), 18 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/regress/expected/vacuum_tables_statistics.out
create mode 100644 src/test/regress/sql/vacuum_tables_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 09fab08b8e1..919f0103e85 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -165,6 +165,7 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -229,6 +230,8 @@ typedef struct LVRelState
BlockNumber next_unskippable_block; /* next unskippable block */
bool next_unskippable_allvis; /* its visibility status */
Buffer next_unskippable_vmbuffer; /* buffer containing its VM bit */
+
+ int64 wraparound_failsafe_count; /* the number of times to prevent workaround problem */
} LVRelState;
/* Struct for saving and restoring vacuum error information. */
@@ -239,6 +242,18 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -292,6 +307,106 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -324,7 +439,14 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -342,7 +464,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -359,6 +481,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -446,6 +569,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->vm_new_visible_pages = 0;
vacrel->vm_new_visible_frozen_pages = 0;
vacrel->vm_new_frozen_pages = 0;
+ vacrel->wraparound_failsafe_count = 0;
/*
* Get cutoffs that determine which deleted tuples are considered DEAD,
@@ -591,6 +715,26 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Fill heap-specific extended stats fields */
+ extVacReport.pages_scanned = vacrel->scanned_pages;
+ extVacReport.pages_removed = vacrel->removed_pages;
+ extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ }
+
/*
* Report results to the cumulative stats system, too.
*
@@ -605,7 +749,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -2418,6 +2563,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[2] = {0, 0};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count ++;
/*
* Abandon use of a buffer access strategy to allow use of all of
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 745a04ef26e..07623a045fa 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -91,6 +91,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -160,6 +161,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * As part of vacuum stats, track how often all-visible or all-frozen
+ * bits are cleared.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 7a595c84db9..43ac27ed5b4 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -691,7 +691,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_vacuum_count(C.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(C.oid) AS autovacuum_count,
pg_stat_get_analyze_count(C.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
@@ -1381,3 +1383,51 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+SELECT
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+ stats.relid as relid,
+
+ stats.total_blks_read AS total_blks_read,
+ stats.total_blks_hit AS total_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.rel_blks_read AS rel_blks_read,
+ stats.rel_blks_hit AS rel_blks_hit,
+
+ stats.pages_scanned AS pages_scanned,
+ stats.pages_removed AS pages_removed,
+ stats.vm_new_frozen_pages AS vm_new_frozen_pages,
+ stats.vm_new_visible_pages AS vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ stats.missed_dead_pages AS missed_dead_pages,
+ stats.tuples_deleted AS tuples_deleted,
+ stats.tuples_frozen AS tuples_frozen,
+ stats.recently_dead_tuples AS recently_dead_tuples,
+ stats.missed_dead_tuples AS missed_dead_tuples,
+
+ stats.wraparound_failsafe AS wraparound_failsafe,
+ stats.index_vacuum_count AS index_vacuum_count,
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time
+
+FROM pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
+WHERE rel.relkind = 'r';
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 2640228bef4..b074bde286d 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -102,6 +102,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2419,6 +2422,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 0d92e694d6a..672f8f4bfe8 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1047,6 +1047,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 34520535d54..d21b9302c29 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -191,7 +191,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
@@ -204,7 +204,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = true;
/* ----------
* state shared with pgstat_*.c
@@ -261,7 +261,6 @@ static bool pgstat_is_initialized = false;
static bool pgstat_is_shutdown = false;
#endif
-
/*
* The different kinds of built-in statistics.
*
@@ -911,7 +910,6 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
-
/* ------------------------------------------------------------
* Fetching of stats
* ------------------------------------------------------------
@@ -980,7 +978,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -1096,7 +1094,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -1147,7 +1145,7 @@ pgstat_prep_snapshot(void)
}
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 09247ba0971..458bd4ece49 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,6 +47,8 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
+static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
@@ -208,7 +210,8 @@ pgstat_drop_relation(Relation rel)
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
@@ -232,6 +235,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -859,6 +864,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
@@ -982,3 +990,40 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
trans->tuples_deleted = trans->deleted_pre_truncdrop;
}
}
+
+static void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->total_time += src->total_time;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ dst->pages_scanned += src->pages_scanned;
+ dst->pages_removed += src->pages_removed;
+ dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
+ dst->vm_new_visible_pages += src->vm_new_visible_pages;
+ dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->tuples_frozen += src->tuples_frozen;
+ dst->recently_dead_tuples += src->recently_dead_tuples;
+ dst->index_vacuum_count += src->index_vacuum_count;
+ dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
+ dst->missed_dead_pages += src->missed_dead_pages;
+ dst->missed_dead_tuples += src->missed_dead_tuples;
+
+}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5f8d20a406d..f180ac0fa02 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_TIMESTAMPTZ(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
@@ -2172,3 +2178,144 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_scanned",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_removed",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "vm_new_visible_frozen_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_pages",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_frozen",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "recently_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "missed_dead_tuples",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wraparound_failsafe_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "index_vacuum_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
\ No newline at end of file
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 860bbd40d42..4da8d3f87fd 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1619,6 +1619,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index c9d8cd796a8..f2d31e174b4 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1478,6 +1478,15 @@ struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
+ gettext_noop("Collects vacuum statistics for table relations."),
+ NULL
+ },
+ &pgstat_track_vacuum_statistics,
+ true,
+ NULL, NULL, NULL
+ },
{
{"track_wal_io_timing", PGC_SUSET, STATS_CUMULATIVE,
gettext_noop("Collects timing statistics for WAL I/O activity."),
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b37e8a6f882..b0e363794dc 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12429,4 +12429,22 @@
proargtypes => 'int2',
prosrc => 'gist_stratnum_identity' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables return stats values',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
+
+ { oid => '8002', descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+ { oid => '8003', descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 12d0b61950d..94d599767df 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 6475889c58c..818350af8d4 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -167,6 +167,53 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a heap relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ /* number of blocks missed, hit, dirtied and written during a vacuum of specific relation */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* blocks missed and hit for just the heap during a vacuum of specific relation */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 index_vacuum_count; /* the number of index vacuumings */
+ int64 wraparound_failsafe_count; /* the number of times to prevent workaround problem */
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -209,6 +256,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -267,7 +324,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB0
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCB1
typedef struct PgStat_ArchiverStats
{
@@ -429,6 +486,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter parallel_workers_launched;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -502,6 +561,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -676,7 +740,8 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
@@ -727,6 +792,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -744,7 +820,6 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
/*
* Functions in pgstat_replslot.c
*/
@@ -815,6 +890,7 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
/*
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 7161f5c6ad6..9e080747a92 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -230,6 +230,7 @@ extern int geterrlevel(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 4bb8e5c53ab..877218723dc 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -600,7 +600,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
*/
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..87f7e40b4a6
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..e93dd4f626c 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -95,6 +95,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..5893d89573d
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,53 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 3014d047fef..dd795d58dfc 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1804,7 +1804,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_vacuum_count(c.oid) AS vacuum_count,
pg_stat_get_autovacuum_count(c.oid) AS autovacuum_count,
pg_stat_get_analyze_count(c.oid) AS analyze_count,
- pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count
+ pg_stat_get_autoanalyze_count(c.oid) AS autoanalyze_count,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2188,7 +2190,9 @@ pg_stat_sys_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2236,9 +2240,42 @@ pg_stat_user_tables| SELECT relid,
vacuum_count,
autovacuum_count,
analyze_count,
- autoanalyze_count
+ autoanalyze_count,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_tables| SELECT ns.nspname AS schema,
+ rel.relname,
+ stats.relid,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.missed_dead_pages,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.missed_dead_tuples,
+ stats.index_vacuum_count,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'r'::"char");
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
new file mode 100644
index 00000000000..0c05a812dd1
--- /dev/null
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -0,0 +1,225 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+ relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_scanned | pages_removed | vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | missed_dead_pages | tuples_deleted | tuples_frozen | recently_dead_tuples | missed_dead_tuples | index_vacuum_count | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+---------------+---------------------+----------------------+-----------------------------+-------------------+----------------+---------------+----------------------+--------------------+--------------------+-------------+---------+-----------+---------------+----------------+------------+------------
+ vestat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
+(1 row)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | 0 | 0 | 455 | 0 | 0
+(1 row)
+
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | f | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+ dwr | dwb
+-----+-----
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+ dwr | dfpi | dwb
+-----+------+-----
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+ relname | vm_new_frozen_pages | tuples_deleted | relpages | pages_scanned | pages_removed
+---------+---------------------+----------------+----------+---------------+---------------
+ vestat | t | t | f | t | t
+(1 row)
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
+---------------------+----------------------+----------------------+-----------------------+-----------------------------
+ 0 | 910 | 0 | 0 | 455
+(1 row)
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | t | t | t
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ f | t | f | f | f
+(1 row)
+
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+ vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
+---------------------+----------------------+-----------------------------+----------------------+-----------------------
+ t | t | t | t | t
+(1 row)
+
+DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 1edd9e45ebb..d11f6b7ef4b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -136,3 +136,8 @@ test: fast_default
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
+
+# ----------
+# Check vacuum statistics
+# ----------
+test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
new file mode 100644
index 00000000000..8ad69108ca1
--- /dev/null
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -0,0 +1,180 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT relname,total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written,rel_blks_read, rel_blks_hit,
+pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages,
+tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, index_vacuum_count,
+wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time,delay_time, total_time
+FROM pg_stat_vacuum_tables vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS roid from pg_class where relname = 'vestat' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,vm_new_frozen_pages,tuples_deleted,relpages,pages_scanned,pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT relpages AS rp
+FROM pg_class c
+WHERE relname = 'vestat' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,vm_new_frozen_pages > 0 AS vm_new_frozen_pages,tuples_deleted > 0 AS tuples_deleted,relpages-:rp = 0 AS relpages,pages_scanned > 0 AS pages_scanned,pages_removed = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS dWR, wal_bytes > 0 AS dWB
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,vm_new_frozen_pages-:fp > 0 AS vm_new_frozen_pages,tuples_deleted-:td > 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps > 0 AS pages_scanned,pages_removed-:pr > 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps, pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr, wal_bytes-:hwb AS dwb, wal_fpi-:hfpi AS dfpi
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL advance should be detected.
+SELECT :dwr > 0 AS dWR, :dwb > 0 AS dWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:hwr AS dwr2, wal_bytes-:hwb AS dwb2, wal_fpi-:hfpi AS dfpi2
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :dwr2=0 AS dWR, :dfpi2=0 AS dFPI, :dwb2=0 AS dWB;
+
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp < 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+SELECT vm_new_frozen_pages AS fp,tuples_deleted AS td,relpages AS rp, pages_scanned AS ps,pages_removed AS pr
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS hwr,wal_bytes AS hwb,wal_fpi AS hfpi FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP OFF) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:hwr AS dwr3, wal_bytes-:hwb AS dwb3, wal_fpi-:hfpi AS dfpi3
+FROM pg_stat_vacuum_tables WHERE relname = 'vestat' \gset
+
+--There are nothing changed
+SELECT :dwr3>0 AS dWR, :dfpi3=0 AS dFPI, :dwb3>0 AS dWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The vm_new_frozen_pages, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_deleted-:td = 0 AS tuples_deleted,relpages -:rp = 0 AS relpages,pages_scanned-:ps = 0 AS pages_scanned,pages_removed-:pr = 0 AS pages_removed
+FROM pg_stat_vacuum_tables vt, pg_class c
+WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+-- must be empty
+SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- backend defreezed pages
+SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+UPDATE vestat SET x = x + 1001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- vacuum freezed pages
+SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
+FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+
+DROP TABLE vestat CASCADE;
--
2.34.1
[text/x-patch] v15-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (61.1K, 4-v15-0002-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From fe6b1cc600f9945a2aaeabe11123ed900ac6077d Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 16 Dec 2024 12:06:17 +0300
Subject: [PATCH 2/4] Machinery for grabbing an extended vacuum statistics on
index relations.
They are gathered separatelly from table statistics.
As for tables, we gather vacuum shared buffers statistics for index relations like
value of total_blks_hit, total_blks_read, total_blks_dirtied, wal statistics, io time
during flushing buffer pages to disk, delay and total time.
Due to the fact taht such statistics are common as for tables, as for indexes we
set them in the union ExtVacReport structure. We only added some determination 'type'
field to highlight what kind belong to these statistics: PGSTAT_EXTVAC_TABLE or
PGSTAT_EXTVAC_INDEX. Generally, PGSTAT_EXTVAC_INVALID type leads to wrong code process.
Some statistics belong only one type of both tables or indexes. So, we added substructures
table and index inside ExtVacReport structure.
Therefore, we gather only for tables such statistics like number of scanned, removed pages,
their charecteristics according VM (all-visible and frozen). In addition, for tables we
gather number frozen, deleted and recently dead tuples and how many times vacuum processed
indexes for tables.
Controversally for indexes we gather number of deleted pages and deleted tuples only.
As for tables, deleted pages and deleted tuples reflect the overall performance of the vacuum
for the index relationship
Since vacuum clears tuple index references before clearing table tuples, we need to subtract
number of collected index vacuum statistics for general statistics taken into account
for the table and for the index, especially for
shared buffers: total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written;
wal statistics: wal_bytes, wal_fpi, wal_records; IO time: blk_read_time, blk_write_time;
total time and delay time. This is necessary to take into account vacuum statistics for tables
and indexes separately.
Due to the fact that the vacuum can produce workers to process indexes of the table and
workers store their wal and buffer statistic information separately from the place that
leader does we need to store an information about ParallelWorkerNumber that's why we
added the variable id_parallel_worker in the index's statistic structure. PVIndStats
stryucture store statistic information for every index whether it was processed by
leader or worker, so we are sure that id_parallel_worker will be updated correctly.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 166 ++++++++++++----
src/backend/catalog/system_views.sql | 32 +++
src/backend/commands/vacuumparallel.c | 59 +++++-
src/backend/utils/activity/pgstat.c | 4 +
src/backend/utils/activity/pgstat_relation.c | 52 +++--
src/backend/utils/adt/pgstatfuncs.c | 133 +++++++++++--
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 9 +
src/include/commands/vacuum.h | 26 +++
src/include/pgstat.h | 61 ++++--
src/include/utils/pgstat_internal.h | 1 -
src/test/regress/expected/rules.out | 22 +++
.../expected/vacuum_index_statistics.out | 183 ++++++++++++++++++
.../expected/vacuum_tables_statistics.out | 40 +++-
src/test/regress/parallel_schedule | 1 +
.../regress/sql/vacuum_index_statistics.sql | 151 +++++++++++++++
.../regress/sql/vacuum_tables_statistics.sql | 12 ++
17 files changed, 864 insertions(+), 90 deletions(-)
create mode 100644 src/test/regress/expected/vacuum_index_statistics.out
create mode 100644 src/test/regress/sql/vacuum_index_statistics.sql
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 919f0103e85..15d27ed39d9 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -166,6 +166,7 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -242,19 +243,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-/*
- * Counters and usage data for extended stats tracking.
- */
-typedef struct LVExtStatCounters
-{
- TimestampTz starttime;
- WalUsage walusage;
- BufferUsage bufusage;
- double VacuumDelayTime;
- PgStat_Counter blocks_fetched;
- PgStat_Counter blocks_hit;
-} LVExtStatCounters;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static bool heap_vac_scan_next_block(LVRelState *vacrel, BlockNumber *blkno,
@@ -352,7 +340,8 @@ extvac_stats_start(Relation rel, LVExtStatCounters *counters)
*/
static void
extvac_stats_end(Relation rel, LVExtStatCounters *counters,
- ExtVacReport *report)
+ ExtVacReport *report, BufferUsage *worker_bufferusage,
+ WalUsage *worker_walusage)
{
WalUsage walusage;
BufferUsage bufusage;
@@ -365,10 +354,16 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
/* Calculate diffs of global stat parameters on WAL and buffer usage. */
memset(&walusage, 0, sizeof(WalUsage));
- WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ if(worker_walusage == NULL)
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ else
+ WalUsageAccumDiff(&walusage, worker_walusage, &counters->walusage);
memset(&bufusage, 0, sizeof(BufferUsage));
- BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ if(worker_bufferusage == NULL)
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ else
+ BufferUsageAccumDiff(&bufusage, worker_bufferusage, &counters->bufusage);
endtime = GetCurrentTimestamp();
TimestampDifference(counters->starttime, endtime, &secs, &usecs);
@@ -408,6 +403,59 @@ extvac_stats_end(Relation rel, LVExtStatCounters *counters,
rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
}
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ BufferUsage *buffusage, WalUsage *walusage)
+{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if(buffusage != NULL)
+ memset(&(counters->common.bufusage), 0, sizeof(BufferUsage));
+
+ if(walusage != NULL)
+ memset(&(counters->common.walusage), 0, sizeof(WalUsage));
+
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report,
+ BufferUsage *buffusage, WalUsage *walusage)
+{
+ extvac_stats_end(rel, &counters->common, report,
+ buffusage, walusage);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -442,11 +490,6 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
LVExtStatCounters extVacCounters;
ExtVacReport extVacReport;
char **indnames = NULL;
- ExtVacReport allzero;
-
- /* Initialize vacuum statistics */
- memset(&allzero, 0, sizeof(ExtVacReport));
- extVacReport = allzero;
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -714,25 +757,36 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
new_rel_allvisible, vacrel->nindexes > 0,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
-
- /* Make generic extended vacuum stats report */
- extvac_stats_end(rel, &extVacCounters, &extVacReport);
+ extvac_stats_end(rel, &extVacCounters, &extVacReport, NULL, NULL);
if(pgstat_track_vacuum_statistics)
{
/* Fill heap-specific extended stats fields */
- extVacReport.pages_scanned = vacrel->scanned_pages;
- extVacReport.pages_removed = vacrel->removed_pages;
- extVacReport.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
- extVacReport.vm_new_visible_pages = vacrel->vm_new_visible_pages;
- extVacReport.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacReport.type = PGSTAT_EXTVAC_TABLE;
+ extVacReport.table.pages_scanned = vacrel->scanned_pages;
+ extVacReport.table.pages_removed = vacrel->removed_pages;
+ extVacReport.table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacReport.table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacReport.table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
extVacReport.tuples_deleted = vacrel->tuples_deleted;
- extVacReport.tuples_frozen = vacrel->tuples_frozen;
- extVacReport.recently_dead_tuples = vacrel->recently_dead_tuples;
- extVacReport.missed_dead_tuples = vacrel->missed_dead_tuples;
- extVacReport.missed_dead_pages = vacrel->missed_dead_pages;
- extVacReport.index_vacuum_count = vacrel->num_index_scans;
- extVacReport.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+ extVacReport.table.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacReport.table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacReport.table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacReport.table.index_vacuum_count = vacrel->num_index_scans;
+ extVacReport.table.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ }
+ else
+ {
+ ExtVacReport allzero;
+
+ /* Initialize vacuum statistics with 0 values to prevent
+ * adding garbage values in memory
+ */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extVacReport = allzero;
}
/*
@@ -2507,6 +2561,7 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->vm_new_frozen_pages++;
}
PageSetAllVisible(page);
@@ -2675,6 +2730,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters, NULL, NULL);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2693,6 +2752,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2701,6 +2761,15 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport, NULL, NULL);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2725,6 +2794,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters, NULL, NULL);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2744,12 +2817,22 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport, NULL, NULL);
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3325,6 +3408,8 @@ update_relstats_all_indexes(LVRelState *vacrel)
Relation *indrels = vacrel->indrels;
int nindexes = vacrel->nindexes;
IndexBulkDeleteResult **indstats = vacrel->indstats;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
Assert(vacrel->do_index_cleanup);
@@ -3336,6 +3421,8 @@ update_relstats_all_indexes(LVRelState *vacrel)
if (istat == NULL || istat->estimated_count)
continue;
+ extvac_stats_start_idx(indrel, istat, &extVacCounters, NULL, NULL);
+
/* Update index statistics */
vac_update_relstats(indrel,
istat->num_pages,
@@ -3345,6 +3432,15 @@ update_relstats_all_indexes(LVRelState *vacrel)
InvalidTransactionId,
InvalidMultiXactId,
NULL, NULL, false);
+
+ if(pgstat_track_vacuum_statistics)
+ {
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport, NULL, NULL);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+ }
}
}
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 43ac27ed5b4..09e93b21e82 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1431,3 +1431,35 @@ FROM pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_tables(rel.oid) stats
WHERE rel.relkind = 'r';
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS schemaname,
+ rel.relname AS relname,
+
+ total_blks_read AS total_blks_read,
+ total_blks_hit AS total_blks_hit,
+ total_blks_dirtied AS total_blks_dirtied,
+ total_blks_written AS total_blks_written,
+
+ rel_blks_read AS rel_blks_read,
+ rel_blks_hit AS rel_blks_hit,
+
+ pages_deleted AS pages_deleted,
+ tuples_deleted AS tuples_deleted,
+
+ wal_records AS wal_records,
+ wal_fpi AS wal_fpi,
+ wal_bytes AS wal_bytes,
+
+ blk_read_time AS blk_read_time,
+ blk_write_time AS blk_write_time,
+
+ delay_time AS delay_time,
+ total_time AS total_time
+FROM
+ pg_class rel
+ JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
+WHERE rel.relkind = 'i';
\ No newline at end of file
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 672f8f4bfe8..c6ee0d63d13 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -154,6 +154,10 @@ typedef struct PVIndStats
*/
bool istat_updated; /* are the stats updated? */
IndexBulkDeleteResult istat;
+
+ LVExtStatCountersIdx counters;
+ ExtVacReport idx_report;
+ int id_parallel_worker; /* detect index was processed by postmster or worker */
} PVIndStats;
/*
@@ -654,6 +658,8 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
{
PVIndStats *indstats = &(pvs->indstats[i]);
+ indstats->id_parallel_worker = -2;
+
Assert(indstats->status == PARALLEL_INDVAC_STATUS_INITIAL);
indstats->status = new_status;
indstats->parallel_workers_can_process =
@@ -661,6 +667,12 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
parallel_vacuum_index_is_parallel_safe(pvs->indrels[i],
num_index_scans,
vacuum));
+
+ /* Sava buffer and wal statistics before vacuuming to track them
+ * for the leader.
+ */
+ extvac_stats_start_idx(pvs->indrels[i], &(indstats->istat),
+ &(indstats->counters),NULL, NULL);
}
/* Reset the parallel index processing and progress counters */
@@ -727,19 +739,10 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
*/
parallel_vacuum_process_safe_indexes(pvs);
- /*
- * Next, accumulate buffer and WAL usage. (This must wait for the workers
- * to finish, or we might get incomplete data.)
- */
if (nworkers > 0)
- {
/* Wait for all vacuum workers to finish */
WaitForParallelWorkersToFinish(pvs->pcxt);
- for (int i = 0; i < pvs->pcxt->nworkers_launched; i++)
- InstrAccumParallelQuery(&pvs->buffer_usage[i], &pvs->wal_usage[i]);
- }
-
/*
* Reset all index status back to initial (while checking that we have
* vacuumed all indexes).
@@ -752,9 +755,44 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
elog(ERROR, "parallel index vacuum on index \"%s\" is not completed",
RelationGetRelationName(pvs->indrels[i]));
+ /* If an index was processed by worker we need to gather wal and
+ * buffer statistics from pvs->buffer_usage and pvs->wal_usage,
+ * otherwice for leader they can be detected through substract
+ * of global statistics of pgWalUsage and pgBufferUsage.
+ */
+ if(pgstat_track_vacuum_statistics)
+ {
+ /* We expect that all indexes have updated it */
+ Assert(indstats->id_parallel_worker != -2);
+
+ if(indstats->id_parallel_worker == -1)
+ extvac_stats_end_idx(pvs->indrels[i], &(indstats->istat), &(indstats->counters),
+ &(indstats->idx_report), NULL, NULL);
+ else
+ {
+ /* We need to reset Buffer and Wal usage statistics */
+ memset(&(indstats->counters.common.bufusage), 0, sizeof(BufferUsage));
+ memset(&(indstats->counters.common.walusage), 0, sizeof(WalUsage));
+ extvac_stats_end_idx(pvs->indrels[i], &(indstats->istat), &(indstats->counters),
+ &(indstats->idx_report), &pvs->buffer_usage[indstats->id_parallel_worker], &pvs->wal_usage[indstats->id_parallel_worker]);
+ }
+
+ pgstat_report_vacuum(RelationGetRelid(pvs->indrels[i]),
+ pvs->indrels[i]->rd_rel->relisshared,
+ 0, 0, &(indstats->idx_report));
+ }
+
indstats->status = PARALLEL_INDVAC_STATUS_INITIAL;
}
+ /*
+ * Next, accumulate buffer and WAL usage. (This must wait for the workers
+ * to finish, or we might get incomplete data.)
+ */
+ if (nworkers > 0)
+ for (int i = 0; i < pvs->pcxt->nworkers_launched; i++)
+ InstrAccumParallelQuery(&pvs->buffer_usage[i], &pvs->wal_usage[i]);
+
/*
* Carry the shared balance value to heap scan and disable shared costing
*/
@@ -925,6 +963,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
pfree(istat_res);
}
+ /* Like ParallelWorkerNumber can be -1 for leader and more 0 for workers */
+ indstats->id_parallel_worker = ParallelWorkerNumber;
+
/*
* Update the status to completed. No need to lock here since each worker
* touches different indexes.
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index d21b9302c29..dc81bb12c86 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -1190,6 +1190,10 @@ pgstat_build_snapshot(PgStat_Kind statKind)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 458bd4ece49..db612d243cd 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -47,8 +47,6 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev
static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info);
static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop);
static void restore_truncdrop_counters(PgStat_TableXactStatus *trans);
-static void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
- bool accumulate_reltype_specific_info);
/*
@@ -991,10 +989,13 @@ restore_truncdrop_counters(PgStat_TableXactStatus *trans)
}
}
-static void
+void
pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
bool accumulate_reltype_specific_info)
{
+ if(!pgstat_track_vacuum_statistics)
+ return;
+
dst->total_blks_read += src->total_blks_read;
dst->total_blks_hit += src->total_blks_hit;
dst->total_blks_dirtied += src->total_blks_dirtied;
@@ -1010,20 +1011,35 @@ pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
if (!accumulate_reltype_specific_info)
return;
- dst->blks_fetched += src->blks_fetched;
- dst->blks_hit += src->blks_hit;
-
- dst->pages_scanned += src->pages_scanned;
- dst->pages_removed += src->pages_removed;
- dst->vm_new_frozen_pages += src->vm_new_frozen_pages;
- dst->vm_new_visible_pages += src->vm_new_visible_pages;
- dst->vm_new_visible_frozen_pages += src->vm_new_visible_frozen_pages;
- dst->tuples_deleted += src->tuples_deleted;
- dst->tuples_frozen += src->tuples_frozen;
- dst->recently_dead_tuples += src->recently_dead_tuples;
- dst->index_vacuum_count += src->index_vacuum_count;
- dst->wraparound_failsafe_count += src->wraparound_failsafe_count;
- dst->missed_dead_pages += src->missed_dead_pages;
- dst->missed_dead_tuples += src->missed_dead_tuples;
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->tuples_deleted += src->tuples_deleted;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.wraparound_failsafe_count += src->table.wraparound_failsafe_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->tuples_deleted += src->tuples_deleted;
+ }
+ }
}
\ No newline at end of file
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index f180ac0fa02..d41687ca39a 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2286,18 +2286,19 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
extvacuum->blks_hit);
values[i++] = Int64GetDatum(extvacuum->blks_hit);
- values[i++] = Int64GetDatum(extvacuum->pages_scanned);
- values[i++] = Int64GetDatum(extvacuum->pages_removed);
- values[i++] = Int64GetDatum(extvacuum->vm_new_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_pages);
- values[i++] = Int64GetDatum(extvacuum->vm_new_visible_frozen_pages);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
- values[i++] = Int64GetDatum(extvacuum->tuples_frozen);
- values[i++] = Int64GetDatum(extvacuum->recently_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->missed_dead_tuples);
- values[i++] = Int64GetDatum(extvacuum->wraparound_failsafe_count);
- values[i++] = Int64GetDatum(extvacuum->index_vacuum_count);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int64GetDatum(extvacuum->table.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
values[i++] = Int64GetDatum(extvacuum->wal_records);
values[i++] = Int64GetDatum(extvacuum->wal_fpi);
@@ -2316,6 +2317,116 @@ pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "relid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "rel_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "pages_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "tuples_deleted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (tabentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(tabentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->blks_fetched -
+ extvacuum->blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f2d31e174b4..2a96cf51a36 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1480,7 +1480,7 @@ struct config_bool ConfigureNamesBool[] =
},
{
{"track_vacuum_statistics", PGC_SUSET, STATS_CUMULATIVE,
- gettext_noop("Collects vacuum statistics for table relations."),
+ gettext_noop("Collects vacuum statistics for relations."),
NULL
},
&pgstat_track_vacuum_statistics,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b0e363794dc..35d6649db31 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12447,4 +12447,13 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes return stats values',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' }
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 94d599767df..c6f5a9ffb02 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -288,6 +289,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -386,4 +407,9 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, BufferUsage *buffusage, WalUsage *walusage);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report,
+ BufferUsage *buffusage, WalUsage *walusage);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 818350af8d4..aef287ba81c 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -167,11 +167,19 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2
+} ExtVacReportType;
+
/* ----------
*
* ExtVacReport
*
- * Additional statistics of vacuum processing over a heap relation.
+ * Additional statistics of vacuum processing over a relation.
* pages_removed is the amount by which the physically shrank,
* if any (ie the change in its total size on disk)
* pages_deleted refer to free space within the index file
@@ -200,18 +208,44 @@ typedef struct ExtVacReport
double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
double total_time; /* total time of a vacuum operation, in msec */
- int64 pages_scanned; /* heap pages examined (not skipped by VM) */
- int64 pages_removed; /* heap pages removed by vacuum "truncation" */
- int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
- int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
- int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
- int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
- int64 missed_dead_pages; /* pages with missed dead tuples */
int64 tuples_deleted; /* tuples deleted by vacuum */
- int64 tuples_frozen; /* tuples frozen up by vacuum */
- int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
- int64 index_vacuum_count; /* the number of index vacuumings */
- int64 wraparound_failsafe_count; /* the number of times to prevent workaround problem */
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* heap pages examined (not skipped by VM) */
+ int64 pages_removed; /* heap pages removed by vacuum "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as all-visible */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are still visible to some transaction */
+ int64 vm_new_frozen_pages; /* pages marked in VM as frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as all-visible and frozen */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due to failure to get a cleanup lock */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ int64 wraparound_failsafe_count; /* the number of times to prevent workaround problem */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
} ExtVacReport;
/* ----------
@@ -734,7 +768,8 @@ extern PgStat_FunctionCounts *find_funcstat_entry(Oid func_id);
extern void pgstat_create_relation(Relation rel);
extern void pgstat_drop_relation(Relation rel);
extern void pgstat_copy_relation_stats(Relation dst, Relation src);
-
+extern void pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
extern void pgstat_init_relation(Relation rel);
extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 877218723dc..4f836e7fca0 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -600,7 +600,6 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
*/
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index dd795d58dfc..2becc7f3885 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2245,6 +2245,28 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time
+ FROM (pg_class rel
+ JOIN pg_namespace ns ON ((ns.oid = rel.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (rel.relkind = 'i'::"char");
pg_stat_vacuum_tables| SELECT ns.nspname AS schema,
rel.relname,
stats.relid,
diff --git a/src/test/regress/expected/vacuum_index_statistics.out b/src/test/regress/expected/vacuum_index_statistics.out
new file mode 100644
index 00000000000..b840a6ed4fe
--- /dev/null
+++ b/src/test/regress/expected/vacuum_index_statistics.out
@@ -0,0 +1,183 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+ track_counts
+--------------
+ on
+(1 row)
+
+\set sample_size 10000
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+DELETE FROM vestat WHERE x % 2 = 0;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+ relid | schema | relname | total_blks_read | total_blks_hit | total_blks_dirtied | total_blks_written | rel_blks_read | rel_blks_hit | pages_deleted | tuples_deleted | wal_records | wal_fpi | wal_bytes | blk_read_time | blk_write_time | delay_time | total_time
+-------+--------+---------+-----------------+----------------+--------------------+--------------------+---------------+--------------+---------------+----------------+-------------+---------+-----------+---------------+----------------+------------+------------
+(0 rows)
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+SHOW track_vacuum_statistics; -- must be on
+ track_vacuum_statistics
+-------------------------
+ on
+(1 row)
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | 30 | 0 | 0
+(1 row)
+
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+ diwr | diwb
+------+------
+ t | t
+(1 row)
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | t | t | t
+(1 row)
+
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+ diwr | ifpi | diwb
+------+------+------
+ t | t | t
+(1 row)
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+ relname | relpages | pages_deleted | tuples_deleted
+-------------+----------+---------------+----------------
+ vestat_pkey | f | t | t
+(1 row)
+
+DROP TABLE vestat;
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_statistics.out
index 0c05a812dd1..119c7abea5f 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_statistics.out
@@ -181,17 +181,39 @@ WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
vestat | t | t | f | t | t
(1 row)
+DROP TABLE vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
-- must be empty
SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
vm_new_frozen_pages | vm_new_visible_pages | rev_all_frozen_pages | rev_all_visible_pages | vm_new_visible_frozen_pages
---------------------+----------------------+----------------------+-----------------------+-----------------------------
- 0 | 910 | 0 | 0 | 455
+ 0 | 0 | 0 | 0 | 0
(1 row)
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
-- backend defreezed pages
SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
@@ -204,16 +226,28 @@ SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv,vm_new_visible_froz
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
UPDATE vestat SET x = x + 1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
vm_new_frozen_pages | vm_new_visible_pages | vm_new_visible_frozen_pages | rev_all_frozen_pages | rev_all_visible_pages
---------------------+----------------------+-----------------------------+----------------------+-----------------------
- f | t | f | f | f
+ f | t | t | t | t
(1 row)
SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_frozen_pages AS pvf, rev_all_frozen_pages AS hafp,rev_all_visible_pages AS havp
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
-- vacuum freezed pages
SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
@@ -222,4 +256,6 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
t | t | t | t | t
(1 row)
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index d11f6b7ef4b..977a87a5b1f 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -140,4 +140,5 @@ test: tablespace
# ----------
# Check vacuum statistics
# ----------
+test: vacuum_index_statistics
test: vacuum_tables_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_index_statistics.sql b/src/test/regress/sql/vacuum_index_statistics.sql
new file mode 100644
index 00000000000..ae146e1d23f
--- /dev/null
+++ b/src/test/regress/sql/vacuum_index_statistics.sql
@@ -0,0 +1,151 @@
+--
+-- Test cumulative vacuum stats system
+--
+-- Check the wall statistics collected during vacuum operation:
+-- number of frozen and visible pages set by vacuum;
+-- number of frozen and visible pages removed by backend.
+-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
+--
+-- conditio sine qua non
+SHOW track_counts; -- must be on
+
+\set sample_size 10000
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
+-- Test that vacuum statistics will be empty when parameter is off.
+SET track_vacuum_statistics TO 'off';
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+DELETE FROM vestat WHERE x % 2 = 0;
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+-- Must be empty.
+SELECT *
+FROM pg_stat_vacuum_indexes vt
+WHERE vt.relname = 'vestat';
+
+RESET track_vacuum_statistics;
+DROP TABLE vestat CASCADE;
+
+SHOW track_vacuum_statistics; -- must be on
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+\set sample_size 10000
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+--SET stats_fetch_consistency = snapshot;
+CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+
+SELECT oid AS ioid from pg_class where relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat WHERE x % 2 = 0;
+-- Before the first vacuum execution extended stats view is empty.
+SELECT vt.relname,relpages,pages_deleted,tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT relpages AS irp
+FROM pg_class c
+WHERE relname = 'vestat_pkey' \gset
+
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- The table and index extended vacuum statistics should show us that
+-- vacuum frozed pages and clean up pages, but pages_removed stayed the same
+-- because of not full table have cleaned up
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted = 0 AS pages_deleted,tuples_deleted > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- Look into WAL records deltas.
+SELECT wal_records > 0 AS diWR, wal_bytes > 0 AS diWB
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey';
+
+DELETE FROM vestat;;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- pages_removed must be increased
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd > 0 AS pages_deleted,tuples_deleted-:itd > 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr, wal_bytes-:iwb AS diwb, wal_fpi-:ifpi AS difpi
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL advance should be detected.
+SELECT :diwr > 0 AS diWR, :diwb > 0 AS diWB;
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+DELETE FROM vestat WHERE x % 2 = 0;
+-- VACUUM FULL doesn't report to stat collector. So, no any advancements of statistics
+-- are detected here.
+VACUUM FULL vestat;
+-- It is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables
+SELECT wal_records-:iwr AS diwr2, wal_bytes-:iwb AS diwb2, wal_fpi-:ifpi AS difpi2
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+-- WAL and other statistics advance should not be detected.
+SELECT :diwr2=0 AS diWR, :difpi2=0 AS iFPI, :diwb2=0 AS diWB;
+
+SELECT vt.relname,relpages-:irp < 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+SELECT vt.relname,relpages AS irp,pages_deleted AS ipd,tuples_deleted AS itd
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid \gset
+
+-- Store WAL advances into variables
+SELECT wal_records AS iwr,wal_bytes AS iwb,wal_fpi AS ifpi FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+DELETE FROM vestat;
+TRUNCATE vestat;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128, INDEX_CLEANUP ON) vestat;
+-- it is necessary to check the wal statistics
+CHECKPOINT;
+
+-- Store WAL advances into variables after removing all tuples from the table
+SELECT wal_records-:iwr AS diwr3, wal_bytes-:iwb AS diwb3, wal_fpi-:ifpi AS difpi3
+FROM pg_stat_vacuum_indexes WHERE relname = 'vestat_pkey' \gset
+
+--There are nothing changed
+SELECT :diwr3=0 AS diWR, :difpi3=0 AS iFPI, :diwb3=0 AS diWB;
+
+--
+-- Now, the table and index is compressed into zero number of pages. Check it
+-- in vacuum extended statistics.
+-- The pages_frozen, pages_scanned values shouldn't be changed
+--
+SELECT vt.relname,relpages-:irp = 0 AS relpages,pages_deleted-:ipd = 0 AS pages_deleted,tuples_deleted-:itd = 0 AS tuples_deleted
+FROM pg_stat_vacuum_indexes vt, pg_class c
+WHERE vt.relname = 'vestat_pkey' AND vt.relid = c.oid;
+
+DROP TABLE vestat;
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_statistics.sql
index 8ad69108ca1..dfd7af70027 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_statistics.sql
@@ -148,14 +148,22 @@ SELECT vt.relname,vm_new_frozen_pages-:fp = 0 AS vm_new_frozen_pages,tuples_dele
FROM pg_stat_vacuum_tables vt, pg_class c
WHERE vt.relname = 'vestat' AND vt.relid = c.oid;
+DROP TABLE vestat;
+SELECT pg_stat_force_next_flush();
+SET vacuum_freeze_min_age = 0;
+SET vacuum_freeze_table_age = 0;
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+
INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
ANALYZE vestat;
+SELECT pg_stat_force_next_flush();
-- must be empty
SELECT vm_new_frozen_pages, vm_new_visible_pages, rev_all_frozen_pages,rev_all_visible_pages,vm_new_visible_frozen_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
-- backend defreezed pages
SELECT vm_new_frozen_pages > 0 AS vm_new_frozen_pages,vm_new_visible_pages > 0 AS vm_new_visible_pages,vm_new_visible_frozen_pages > 0 AS vm_new_visible_frozen_pages,rev_all_frozen_pages = 0 AS rev_all_frozen_pages,rev_all_visible_pages = 0 AS rev_all_visible_pages
@@ -165,6 +173,7 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
UPDATE vestat SET x = x + 1001;
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
SELECT vm_new_frozen_pages > :pf AS vm_new_frozen_pages,vm_new_visible_pages > :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages > :pvf AS vm_new_visible_frozen_pages,rev_all_frozen_pages > :hafp AS rev_all_frozen_pages,rev_all_visible_pages > :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
@@ -172,9 +181,12 @@ SELECT vm_new_frozen_pages AS pf, vm_new_visible_pages AS pv, vm_new_visible_fro
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid \gset
VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+SELECT pg_stat_force_next_flush();
-- vacuum freezed pages
SELECT vm_new_frozen_pages = :pf AS vm_new_frozen_pages,vm_new_visible_pages = :pv AS vm_new_visible_pages,vm_new_visible_frozen_pages = :pvf AS vm_new_visible_frozen_pages, rev_all_frozen_pages = :hafp AS rev_all_frozen_pages,rev_all_visible_pages = :havp AS rev_all_visible_pages
FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relname = 'vestat' and pg_stat_vacuum_tables.relid = pg_stat_all_tables.relid;
+RESET vacuum_freeze_min_age;
+RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
--
2.34.1
[text/x-patch] v15-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch (21.9K, 5-v15-0003-Machinery-for-grabbing-an-extended-vacuum-statistics.patch)
download | inline diff:
From a50026cefa8cc189b09ea1a81eb1434f9622a7e0 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Fri, 10 Jan 2025 10:50:00 +0300
Subject: [PATCH 3/4] Machinery for grabbing an extended vacuum statistics on
databases.
Database vacuum statistics information is the collected general
vacuum statistics indexes and tables owned by the databases, which
they belong to.
So, buffer, wal, statistics of I/O time of read and writen blocks
statistics will be observed because they are collected for both
tables, indexes. In addition, we show the number of errors caught
during operation of the vacuum only for the error level.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 15 +++
src/backend/catalog/system_views.sql | 26 ++++-
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 43 +++++++++
src/backend/utils/adt/pgstatfuncs.c | 95 +++++++++++++++++++
src/include/catalog/pg_proc.dat | 15 ++-
src/include/pgstat.h | 3 +
src/test/regress/expected/rules.out | 16 ++++
...ut => vacuum_tables_and_db_statistics.out} | 69 +++++++++++++-
src/test/regress/parallel_schedule | 2 +-
...ql => vacuum_tables_and_db_statistics.sql} | 60 +++++++++++-
11 files changed, 338 insertions(+), 7 deletions(-)
rename src/test/regress/expected/{vacuum_tables_statistics.out => vacuum_tables_and_db_statistics.out} (84%)
rename src/test/regress/sql/{vacuum_tables_statistics.sql => vacuum_tables_and_db_statistics.sql} (84%)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 15d27ed39d9..ca1721977e4 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -3458,6 +3458,9 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3473,6 +3476,9 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3488,16 +3494,25 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
+
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() == ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_TABLE);
+
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 09e93b21e82..2f9c22459fa 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1462,4 +1462,28 @@ FROM
pg_class rel
JOIN pg_namespace ns ON ns.oid = rel.relnamespace,
LATERAL pg_stat_get_vacuum_indexes(rel.oid) stats
-WHERE rel.relkind = 'i';
\ No newline at end of file
+WHERE rel.relkind = 'i';
+
+CREATE VIEW pg_stat_vacuum_database AS
+SELECT
+ db.oid as dboid,
+ db.datname AS dbname,
+
+ stats.db_blks_read AS db_blks_read,
+ stats.db_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS total_blks_dirtied,
+ stats.total_blks_written AS total_blks_written,
+
+ stats.wal_records AS wal_records,
+ stats.wal_fpi AS wal_fpi,
+ stats.wal_bytes AS wal_bytes,
+
+ stats.blk_read_time AS blk_read_time,
+ stats.blk_write_time AS blk_write_time,
+
+ stats.delay_time AS delay_time,
+ stats.total_time AS total_time,
+ stats.errors AS errors
+FROM
+ pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats;
\ No newline at end of file
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 05a8ccfdb75..d5c1e2a2cf5 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -449,6 +449,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index db612d243cd..7d95a4496d6 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -203,6 +203,38 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.type = m_type;
+ pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.errors++;
+ dbentry->vacuum_ext.type = m_type;
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
@@ -214,6 +246,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -268,6 +301,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
pgstat_flush_io(false);
pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
+
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index d41687ca39a..60a3b672adb 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2427,6 +2427,101 @@ pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ #define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 13
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatDBEntry *dbentry;
+ ExtVacReport *extvacuum;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+ ExtVacReport allzero;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "dbid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_ blks_read",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_hit",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_dirtied",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_blks_written",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_records",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_fpi",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "wal_bytes",
+ NUMERICOID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_read_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "blk_write_time",
+ FLOAT8OID, -1, 0);
+
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "delay_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "total_time",
+ FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) ++i, "errors",
+ INT4OID, -1, 0);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ BlessTupleDesc(tupdesc);
+
+ dbentry = pgstat_fetch_stat_dbentry(dbid);
+
+ if (dbentry == NULL)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(ExtVacReport));
+ extvacuum = &allzero;
+ }
+ else
+ {
+ extvacuum = &(dbentry->vacuum_ext);
+ }
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->wal_records);
+ values[i++] = Int64GetDatum(extvacuum->wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->delay_time);
+ values[i++] = Float8GetDatum(extvacuum->total_time);
+ values[i++] = Float8GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 35d6649db31..a97e3b7a51a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12430,7 +12430,7 @@
prosrc => 'gist_stratnum_identity' },
{ oid => '8001',
- descr => 'pg_stat_get_vacuum_tables return stats values',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
@@ -12448,12 +12448,21 @@
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
{ oid => '8004',
- descr => 'pg_stat_get_vacuum_indexes return stats values',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
proretset => 't',
proargtypes => 'oid',
proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
- prosrc => 'pg_stat_get_vacuum_indexes' }
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index aef287ba81c..5418ece53b5 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -210,6 +210,8 @@ typedef struct ExtVacReport
int64 tuples_deleted; /* tuples deleted by vacuum */
+ int32 errors;
+
ExtVacReportType type; /* heap, index, etc. */
/* ----------
@@ -780,6 +782,7 @@ extern void pgstat_report_vacuum(Oid tableoid, bool shared,
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2becc7f3885..9d84fba378c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2245,6 +2245,22 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT db.oid AS dboid,
+ db.datname AS dbname,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.errors
+ FROM pg_database db,
+ LATERAL pg_stat_get_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, errors);
pg_stat_vacuum_indexes| SELECT rel.oid AS relid,
ns.nspname AS schema,
rel.relname,
diff --git a/src/test/regress/expected/vacuum_tables_statistics.out b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
similarity index 84%
rename from src/test/regress/expected/vacuum_tables_statistics.out
rename to src/test/regress/expected/vacuum_tables_and_db_statistics.out
index 119c7abea5f..5efe8998abe 100644
--- a/src/test/regress/expected/vacuum_tables_statistics.out
+++ b/src/test/regress/expected/vacuum_tables_and_db_statistics.out
@@ -6,7 +6,6 @@
-- number of frozen and visible pages removed by backend.
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
track_counts
--------------
@@ -43,6 +42,11 @@ SHOW track_vacuum_statistics; -- must be on
on
(1 row)
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
@@ -259,3 +263,66 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
RESET vacuum_freeze_min_age;
RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+RESET track_vacuum_statistics;
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+\c regression_statistic_vacuum_db1;
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+ dbname | db_blks_hit | total_blks_dirtied | total_blks_written | wal_records | wal_fpi | wal_bytes | total_time
+--------------------------------+-------------+--------------------+--------------------+-------------+---------+-----------+------------
+ regression_statistic_vacuum_db | t | t | t | t | t | t | t
+(1 row)
+
+\c regression_statistic_vacuum_db
+DROP TABLE vestat CASCADE;
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+ count
+-------
+ 0
+(1 row)
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 977a87a5b1f..19c76b96830 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -141,4 +141,4 @@ test: tablespace
# Check vacuum statistics
# ----------
test: vacuum_index_statistics
-test: vacuum_tables_statistics
\ No newline at end of file
+test: vacuum_tables_and_db_statistics
\ No newline at end of file
diff --git a/src/test/regress/sql/vacuum_tables_statistics.sql b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
similarity index 84%
rename from src/test/regress/sql/vacuum_tables_statistics.sql
rename to src/test/regress/sql/vacuum_tables_and_db_statistics.sql
index dfd7af70027..d0e4a2014c6 100644
--- a/src/test/regress/sql/vacuum_tables_statistics.sql
+++ b/src/test/regress/sql/vacuum_tables_and_db_statistics.sql
@@ -7,7 +7,6 @@
-- Statistic wal_fpi is not displayed in this test because its behavior is unstable.
--
--- conditio sine qua non
SHOW track_counts; -- must be on
\set sample_size 10000
@@ -38,6 +37,13 @@ DROP TABLE vestat CASCADE;
SHOW track_vacuum_statistics; -- must be on
+CREATE DATABASE regression_statistic_vacuum_db;
+CREATE DATABASE regression_statistic_vacuum_db1;
+\c regression_statistic_vacuum_db;
+
+-- not enabled by default, but we want to test it...
+SET track_functions TO 'all';
+
-- ensure pending stats are flushed
SELECT pg_stat_force_next_flush();
@@ -190,3 +196,55 @@ FROM pg_stat_vacuum_tables, pg_stat_all_tables WHERE pg_stat_vacuum_tables.relna
RESET vacuum_freeze_min_age;
RESET vacuum_freeze_table_age;
DROP TABLE vestat CASCADE;
+-- Now check vacuum statistics for current database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = current_database();
+
+RESET track_vacuum_statistics;
+
+-- ensure pending stats are flushed
+SELECT pg_stat_force_next_flush();
+
+CREATE TABLE vestat (x int) WITH (autovacuum_enabled = off, fillfactor = 10);
+INSERT INTO vestat SELECT x FROM generate_series(1,:sample_size) as x;
+ANALYZE vestat;
+UPDATE vestat SET x = 10001;
+VACUUM (PARALLEL 0, BUFFER_USAGE_LIMIT 128) vestat;
+
+\c regression_statistic_vacuum_db1;
+
+-- Now check vacuum statistics for postgres database from another database
+SELECT dbname,
+ db_blks_hit > 0 AS db_blks_hit,
+ total_blks_dirtied > 0 AS total_blks_dirtied,
+ total_blks_written > 0 AS total_blks_written,
+ wal_records > 0 AS wal_records,
+ wal_fpi > 0 AS wal_fpi,
+ wal_bytes > 0 AS wal_bytes,
+ total_time > 0 AS total_time
+FROM
+pg_stat_vacuum_database
+WHERE dbname = 'regression_statistic_vacuum_db';
+
+\c regression_statistic_vacuum_db
+
+DROP TABLE vestat CASCADE;
+
+\c regression_statistic_vacuum_db1;
+SELECT count(*)
+FROM pg_database d
+CROSS JOIN pg_stat_get_vacuum_tables(0)
+WHERE oid = 0; -- must be 0
+
+\c postgres
+DROP DATABASE regression_statistic_vacuum_db1;
+DROP DATABASE regression_statistic_vacuum_db;
--
2.34.1
[text/x-patch] v15-0004-Add-documentation-about-the-system-views-that-are-us.patch (24.2K, 6-v15-0004-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From b359df4cd8928aa455ac9455489f79f21963e997 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 4/4] Add documentation about the system views that are used in
the machinery of vacuum statistics.
---
doc/src/sgml/system-views.sgml | 746 +++++++++++++++++++++++++++++++++
1 file changed, 746 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index a586156614d..73b9e0ae45d 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5066,4 +5066,750 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.34.1
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-10 14:51 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2025-01-10 14:51 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
Hi! I thought about this problem again and I think I have a solution.
On 02.12.2024 23:12, Alena Rybakina wrote:
>> * I've read the previous discussion on how important to keep all these
>> fields regarding vacuum statistics including points by Andrei and Jim.
>> It still worrying me that statistics volume is going to burst in about
>> 3 times, but I don't have a particular proposal on how to make more
>> granular approach. I wonder if you could propose something.
We can collect statistics on databases at all times - there are less
compared to vacuum statistics of relations, but they can give enough
information that can hint that something is going wrong.
With the track_vacuum_statistics guc we can cover cases of collecting
extended and complete information: when it is enabled, we will collect
vacuum statistics on relations both: heaps and indexes.
This will not lead to a synchronicity between constant database
statistics and temporary statistics of relations, since our vacuum
statistics are cumulative and it is assumed that we will look at changes
in statistics over a certain period.
What do you think?
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-10 15:31 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 0 replies; 46+ messages in thread
From: Alena Rybakina @ 2025-01-10 15:31 UTC (permalink / raw)
To: Sami Imseih <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Alexander Korotkov <[email protected]>; Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
Sorry, I made a typo due to lack of sleep, I've marked below where
exactly just in case.
On 10.01.2025 15:04, Alena Rybakina wrote:
>
> Hi, I have updated the patch. Fix minor mistakes in the document,
> added the wraparound_failsafe_count statistics - it accounts the
> number of times when the vacuum operates heap relation to prevent
> workaround problem, fixed "shemaname".
>
I didn't mean workaround problem but wraparound problem.
>>> Secondly, where to put the total time of vacuum for indexes and databases?
>>> It would be incorrect not to take them into account at all. What if we remove the total time from
>>> the heap statistics and add it to pg_stat_tables and only leave the vacuum statistics total time of
>>> vacuum operation of indexes and databases?
>>> It seems strange to me that they will have to be viewed from different views.
>>>
>>> I think it is necessary to look at the total time for tables into perspective of how much
>>> time vacuum spent in total on processing indexes, since indexes can be bloated, for example.
>>> I think it is better to leave these statistics here.
>> You make valid points. I now think because track_vacuum_statistics is
>> optional, we should track total_time in 2 places. First place in the new
>> view being proposed here and the second place is in pg_stat_all_tables
>> as being proposed here [3]. This way if track_vacuum_statistics is off, the
>> total_time of vacuum could still be tracked by pg_stat_all_tables.
>>
>> By the way, the current patch does not track materialized view,
>> but it should as materialized views can also be vacuumed.
>>
>> Regards,
>>
>> Sami
>>
>> [1]https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-VACUUM-FAILSAFE-AGE
>> [2]https://github.com/postgres/postgres/blob/master/src/backend/access/heap/vacuumlazy.c#L2437-L2444
>> [3]https://commitfest.postgresql.org/52/5485/
>>
>>
> I don't agree with this.
>
> Firstly, the hook is enabled by default, that is, it must be specially
> disabled so that the vacuum statistics are not collected.
>
> Secondly, it will cause confusion. First, the hook was disabled and
> statistics were collected in one place - pg_stat_all_tables, and then
> it was enabled and the user notices that the statistics there stopped
> accumulating,
> he is in a panic, "suddenly the vacuum does not work, what to do?".
> The second point here bothers me, how to take into account this
> statistics with the current detailed vacuum statistics? After all,
> adding these values is wrong -
> they do not show the correct statistics regarding the same pages
> processed by vacuum, ignoring it later means that they will be
> redundant. I think it is better to save it here, since this will save
> us from possible confusion.
>
> Secondly, it will immediately show other important parameters
> regarding this statistics - how long the vacuum was sleep (delay_time
> in my patches), how much time the vacuum spent on processing indexes
> during its processing.
> Without this information, this assessment will not be voluminous and
> indicative enough.
>
I didn't mean hook but guc here.
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-13 08:54 Andrei Zubkov <[email protected]>
parent: Sami Imseih <[email protected]>
1 sibling, 0 replies; 46+ messages in thread
From: Andrei Zubkov @ 2025-01-13 08:54 UTC (permalink / raw)
To: Sami Imseih <[email protected]>; Alena Rybakina <[email protected]>; +Cc: Ilia Evdokimov <[email protected]>; Jim Nasby <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]; Alexander Korotkov <[email protected]>
Hi Sami,
Thank you for your attention to our patch and for your own work.
On Sun, 2025-01-05 at 20:00 -0600, Sami Imseih wrote:
>
> You make valid points. I now think because track_vacuum_statistics is
> optional, we should track total_time in 2 places. First place in the
> new
> view being proposed here and the second place is in
> pg_stat_all_tables
> as being proposed here [3]. This way if track_vacuum_statistics is
> off, the
> total_time of vacuum could still be tracked by pg_stat_all_tables.
I think that field total_time in pg_stat_all_tables is redundant at
least if it will be the only field we want to add there. Yes, we have
vacuum counts in pg_stat_all_tables, but those are not related to the
vacuum workload actually. When we think we see unusual numbers there,
we can answer the question "why" - we know the conditions causing
autovacuum to launch a vacuum on every particular table, we have tuple
statistics on this table, and we can detect anomalies here. For
example, when vacuum process should be launched 5 times, but was
launched only twice.
The total_time field is workload metric. Yes, we can calculate the
mean time of vacuum operation on every particular table but there is
nothing we can do with it. We don't know what this time should be for
this table now. We only can compare this metric to its values in the
past. But once we see this time raising we will immediately face the
question "why?". And we have nothing to say about it. Where the time
was spent: vacuuming heap, vacuuming indexes, sleeping in the delay
point or performing IO operations, is there actual workload performed
by vacuum increased with total_time, or now we are spending more time
for the same workload? I think if we are adding workload statistics to
the Cumulative Statistics System we should do it as complete as
possible.
--
Regards, Andrei Zubkov
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2025-01-13 11:29 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 0 replies; 46+ messages in thread
From: Alena Rybakina @ 2025-01-13 11:29 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; +Cc: Jim Nasby <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; pgsql-hackers; [email protected]
On 10.01.2025 17:51, Alena Rybakina wrote:
>
> Hi! I thought about this problem again and I think I have a solution.
>
> On 02.12.2024 23:12, Alena Rybakina wrote:
>>> * I've read the previous discussion on how important to keep all these
>>> fields regarding vacuum statistics including points by Andrei and Jim.
>>> It still worrying me that statistics volume is going to burst in about
>>> 3 times, but I don't have a particular proposal on how to make more
>>> granular approach. I wonder if you could propose something.
> We can collect statistics on databases at all times - there are less
> compared to vacuum statistics of relations, but they can give enough
> information that can hint that something is going wrong.
> With the track_vacuum_statistics guc we can cover cases of collecting
> extended and complete information: when it is enabled, we will collect
> vacuum statistics on relations both: heaps and indexes.
> This will not lead to a synchronicity between constant database
> statistics and temporary statistics of relations, since our vacuum
> statistics are cumulative and it is assumed that we will look at
> changes in statistics over a certain period.
> What do you think?
I implemented this in my latest patch version [0].
[0]
https://www.postgresql.org/message-id/1e81a0a1-a63b-48fb-905a-d6495f89ab73%40postgrespro.ru
--
Regards,
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-02-28 22:20 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2026-02-28 22:20 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 21.12.2025 04:36, Alena Rybakina wrote:
>
> Hi,
> I’ve added some changes to one of the approaches and also did
> additional cleanup and stabilization work on the vacuum statistics
> tests. Specifically:
>
> * I moved the vacuum statistics tests into the tests tab and made
> them more stable. For slower machines, vacuum is now triggered
> inside the statistics wait function. Previously, some backends
> didn’t have enough time to release the lock, which could lead to
> differences because the vacuum hadn’t fully completed yet.
> * I also ran the backend tests and fixed a couple of minor issues
> along the way.
> * I ran pgindent to clean up and normalize the formatting.
>
I have rebased the patch.
I combined all the patches for implementing statistics on tables,
indexes, and databases into one, creating a separate slot for them to
collect and save statistics.
> At the moment, only the second test still looks odd, and I haven’t
> fully figured out why yet. It seems like aggressive vacuum can no
> longer be triggered the same way as before with the current gucs, but
> I’m still investigating this.
>
I fixed this problem - it works well now.
Best regards,
Alena Rybakina
From f1efce79f0aa61c650ec7cc6f40bf6a0806d29f5 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 18:30:12 +0300
Subject: [PATCH 1/3] Introduce new statistics tracking the number of times the
all-visible and all-frozen bits are cleared in the visibility map
(rev_all_visible_pages and rev_all_frozen_pages). These counters, together
with the existing per-vacuum frozen page statistics (vm_new_frozen_pages,
vm_new_visible_pages), help assess how aggressively vacuum is configured and
how frequently the backend has to revoke all-frozen/all-visible bits due to
concurrent modifications.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/visibilitymap.c | 10 ++++++++++
src/backend/catalog/system_views.sql | 4 +++-
src/backend/utils/activity/pgstat_relation.c | 2 ++
src/backend/utils/adt/pgstatfuncs.c | 6 ++++++
src/include/catalog/pg_proc.dat | 12 +++++++++++-
src/include/pgstat.h | 18 +++++++++++++++++-
src/test/regress/expected/rules.out | 12 +++++++++---
7 files changed, 58 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 3047bd46def..2e7c28ea307 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -161,6 +162,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 7553f31fef0..fa102f9c270 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -727,7 +727,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..885d590d2b2 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 73ca0bb0b7f..901f3dd55a1 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5e5e33f64fc..961337ce282 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12693,6 +12693,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index fff7ecc2533..072065adc90 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -156,6 +156,9 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -214,7 +217,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -463,6 +466,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autoanalyze_time;
TimestampTz stat_reset_time;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_StatTabEntry;
/* ------
@@ -722,6 +727,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f4ee2bd7459..7d26bc1a1dc 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2268,7 +2270,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2323,7 +2327,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
From 20e1e8f82e980f31e6855820945067b7268a4962 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 19:55:30 +0300
Subject: [PATCH 2/3] Add machinery for grabbing an extended vacuum statistics.
Vacuum statistics are stored separately from regular relation and
database statistics. Dedicated PGSTAT_KIND_VACUUM_RELATION and
PGSTAT_KIND_VACUUM_DB entries in the cumulative statistics system
allocate memory for vacuum-specific metrics, reducing overall memory use.
Statistics are gathered separately for tables and indexes according to
vacuum phases. The ExtVacReport union and type field distinguish
PGSTAT_EXTVAC_TABLE vs PGSTAT_EXTVAC_INDEX. Heap vacuum stats are sent
to the cumulative statistics system after vacuum has processed the
indexes, according to vacuum phases. Database vacuum statistics
aggregate statistics of tables and indexes in the database.
Common for tables, indexes, and database:
total_blks_hit, total_blks_read, total_blks_dirtied are number of hit,
missed and dirtied pages in shared buffers during a vacuum operation
respectively. total_blks_dirtied means 'dirtied only by this action',
so if a page was dirty before the vacuum operation, it does not count
as dirtied.
blk_read_time and blk_write_time track only access to buffer pages and
flushing them to disk. During vacuum, write time can remain zero if no
flushing operations were performed. total_time is the diff between
timestamps at start and finish; it includes idle time (IO and lock
waits), so it is not equal to the sum of user and system time.
delay_time means total vacuum sleep time in vacuum delay point.
Table and index:
tuples_deleted is the number of tuples cleaned up by the vacuum
operation.
pages_removed is the number of pages by which the physical data storage
of the relation was reduced. pages_deleted is the number of freed pages
in the table (file size may not have changed).
They are processed as independent structures that do not affect each
other, unlike WAL and buffers, so they cannot be summed for the
database as the common statistics above.
Table only:
pages_frozen is the number of pages marked as frozen in VM during
vacuum; it is incremented when a page is marked all-frozen.
pages_all_visible is the number of pages marked as all-visible in VM.
wraparound_failsafe_count is the number of times vacuum started urgent
cleanup to prevent wraparound.
Table and database:
wraparound_failsafe is the count of urgent anti-wraparound cleanups.
Database only:
errors is the number of errors at error level caught during vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 316 +++++++-
src/backend/catalog/heap.c | 1 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/system_views.sql | 109 +++
src/backend/commands/dbcommands.c | 1 +
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 10 +
src/backend/utils/activity/Makefile | 1 +
src/backend/utils/activity/pgstat.c | 30 +-
src/backend/utils/activity/pgstat_database.c | 9 +
src/backend/utils/activity/pgstat_relation.c | 6 +
src/backend/utils/activity/pgstat_vacuum.c | 217 ++++++
src/backend/utils/adt/pgstatfuncs.c | 218 ++++++
src/backend/utils/misc/guc_parameters.dat | 6 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 27 +
src/include/commands/vacuum.h | 26 +
src/include/pgstat.h | 134 +++-
src/include/utils/pgstat_internal.h | 15 +
src/include/utils/pgstat_kind.h | 4 +-
.../vacuum-extending-in-repetable-read.out | 53 ++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 59 ++
.../t/052_vacuum_extending_basic_test.pl | 737 ++++++++++++++++++
.../t/053_vacuum_extending_freeze_test.pl | 329 ++++++++
src/test/regress/expected/rules.out | 75 ++
26 files changed, 2380 insertions(+), 10 deletions(-)
create mode 100644 src/backend/utils/activity/pgstat_vacuum.c
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/recovery/t/052_vacuum_extending_basic_test.pl
create mode 100644 src/test/recovery/t/053_vacuum_extending_freeze_test.pl
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 4be267ff657..60db695b8f0 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -280,6 +280,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -399,6 +401,12 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to
+ * prevent anti-wraparound
+ * shutdown */
+
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -410,7 +418,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static void heap_vacuum_eager_scan_setup(LVRelState *vacrel,
@@ -484,6 +491,227 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ TimestampTz starttime;
+
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written += bufusage.shared_blks_written;
+
+ report->wal_records += walusage.wal_records;
+ report->wal_fpi += walusage.wal_fpi;
+ report->wal_bytes += walusage.wal_bytes;
+
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time += secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched +=
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit +=
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
+
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Set initial values for common heap and index statistics */
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ extvac_stats_end(rel, &counters->common, &report->common);
+
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->common.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+
+}
+
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
/*
@@ -635,7 +863,6 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
new_rel_allvisible,
new_rel_allfrozen;
PGRUsage ru0;
- TimestampTz starttime = 0;
PgStat_Counter startreadtime = 0,
startwritetime = 0;
WalUsage startwalusage = pgWalUsage;
@@ -643,6 +870,11 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ /* Initialize vacuum statistics */
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -657,8 +889,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
}
}
- /* Used for instrumentation and stats report */
- starttime = GetCurrentTimestamp();
+ extvac_stats_start(rel, &extVacCounters);
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
@@ -671,6 +902,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
pgstat_progress_update_param(PROGRESS_VACUUM_STARTED_BY,
PROGRESS_VACUUM_STARTED_BY_MANUAL);
+ extvac_stats_start(rel, &extVacCounters);
+
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -687,6 +920,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -695,6 +929,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
+
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
vac_open_indexes(vacrel->rel, RowExclusiveLock, &vacrel->nindexes,
@@ -797,6 +1034,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->aggressive = vacuum_get_cutoffs(rel, params, &vacrel->cutoffs);
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ vacrel->wraparound_failsafe_count = 0;
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
@@ -959,6 +1197,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ /* extvac_stats_end(rel, &extVacCounters, &extVacReport.common); */
+
/*
* Report results to the cumulative stats system, too.
*
@@ -969,11 +1210,19 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
+
+ /*
+ * Make generic extended vacuum stats report and fill heap-specific
+ * extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+ pgstat_report_vacuum_extstats(vacrel->reloid, rel->rd_rel->relisshared, &extVacReport);
pgstat_report_vacuum(rel,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime);
+ extVacCounters.starttime);
pgstat_progress_end_command();
if (instrument)
@@ -981,7 +1230,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
TimestampTz endtime = GetCurrentTimestamp();
if (verbose || params.log_vacuum_min_duration == 0 ||
- TimestampDifferenceExceeds(starttime, endtime,
+ TimestampDifferenceExceeds(extVacCounters.starttime, endtime,
params.log_vacuum_min_duration))
{
long secs_dur;
@@ -997,7 +1246,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
int64 total_blks_read;
int64 total_blks_dirtied;
- TimestampDifference(starttime, endtime, &secs_dur, &usecs_dur);
+ TimestampDifference(extVacCounters.starttime, endtime, &secs_dur, &usecs_dur);
memset(&walusage, 0, sizeof(WalUsage));
WalUsageAccumDiff(&walusage, &pgWalUsage, &startwalusage);
memset(&bufferusage, 0, sizeof(BufferUsage));
@@ -1940,6 +2189,7 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
/* Count the newly all-frozen pages for logging */
vacrel->vm_new_visible_pages++;
+ vacrel->vm_new_frozen_pages++;
vacrel->vm_new_visible_frozen_pages++;
}
@@ -2662,10 +2912,20 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
* that parallel VACUUM only gets the precheck and this postcheck.
@@ -3012,6 +3272,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3094,10 +3355,20 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
estimated_count);
+
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
/* Reset the progress counters */
@@ -3123,6 +3394,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3141,6 +3420,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3149,6 +3429,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3173,6 +3461,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3192,12 +3485,21 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 606434823cf..024946766d8 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1883,6 +1883,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 43de42ce39e..f006287cbf5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2325,6 +2325,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index fa102f9c270..c9d19c2fb18 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1454,3 +1454,112 @@ REVOKE ALL ON pg_aios FROM PUBLIC;
GRANT SELECT ON pg_aios TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_aios() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+ SELECT
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ S.relid as relid,
+
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
+
+ S.pages_scanned AS pages_scanned,
+ S.pages_removed AS pages_removed,
+ S.vm_new_frozen_pages AS vm_new_frozen_pages,
+ S.vm_new_visible_pages AS vm_new_visible_pages,
+ S.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ S.missed_dead_pages AS missed_dead_pages,
+ S.tuples_deleted AS tuples_deleted,
+ S.tuples_frozen AS tuples_frozen,
+ S.recently_dead_tuples AS recently_dead_tuples,
+ S.missed_dead_tuples AS missed_dead_tuples,
+
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.index_vacuum_count AS index_vacuum_count,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+
+ FROM pg_class C JOIN
+ pg_namespace N ON N.oid = C.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(C.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+ SELECT
+ C.oid AS relid,
+ I.oid AS indexrelid,
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ I.relname AS indexrelname,
+
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
+
+ S.pages_deleted AS pages_deleted,
+ S.tuples_deleted AS tuples_deleted,
+
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+ FROM
+ pg_class C JOIN
+ pg_index X ON C.oid = X.indrelid JOIN
+ pg_class I ON I.oid = X.indexrelid
+ LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace),
+ LATERAL pg_stat_get_vacuum_indexes(I.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
+
+CREATE VIEW pg_stat_vacuum_database AS
+ SELECT
+ D.oid as dboid,
+ D.datname AS dbname,
+
+ S.db_blks_read AS db_blks_read,
+ S.db_blks_hit AS db_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time,
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.errors AS errors
+ FROM
+ pg_database D,
+ LATERAL pg_stat_get_vacuum_database(D.oid) S;
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 87949054f26..9f10710636b 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1815,6 +1815,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 03932f45c8a..769d0ba543f 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -117,6 +117,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2536,6 +2539,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index c3b3c9ea21a..16d215150cc 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,10 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared, &extVacReport);
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1054,6 +1063,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 0eb29ee78aa..df012ae3db5 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 11bb71cad5a..a854a64f135 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = false;
/* ----------
* state shared with pgstat_*.c
@@ -482,6 +482,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index d7f6d4c5ee6..079f02b3f03 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 885d590d2b2..9516552a43e 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -902,6 +902,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
diff --git a/src/backend/utils/activity/pgstat_vacuum.c b/src/backend/utils/activity/pgstat_vacuum.c
new file mode 100644
index 00000000000..d3426e617a8
--- /dev/null
+++ b/src/backend/utils/activity/pgstat_vacuum.c
@@ -0,0 +1,217 @@
+#include "postgres.h"
+
+#include "pgstat.h"
+#include "utils/pgstat_internal.h"
+#include "utils/memutils.h"
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+bool pgstat_track_vacuum_statistics_for_relations = false;
+
+#define ACCUMULATE_FIELD(field) dst->field += src->field;
+
+#define ACCUMULATE_SUBFIELD(substruct, field) \
+ (dst->substruct.field += src->substruct.field)
+
+static void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUMULATE_FIELD(total_blks_read);
+ ACCUMULATE_FIELD(total_blks_hit);
+ ACCUMULATE_FIELD(total_blks_dirtied);
+ ACCUMULATE_FIELD(total_blks_written);
+
+ ACCUMULATE_FIELD(blks_fetched);
+ ACCUMULATE_FIELD(blks_hit);
+
+ ACCUMULATE_FIELD(wal_records);
+ ACCUMULATE_FIELD(wal_fpi);
+ ACCUMULATE_FIELD(wal_bytes);
+
+ ACCUMULATE_FIELD(blk_read_time);
+ ACCUMULATE_FIELD(blk_write_time);
+ ACCUMULATE_FIELD(delay_time);
+ ACCUMULATE_FIELD(total_time);
+
+ ACCUMULATE_FIELD(tuples_deleted);
+ ACCUMULATE_FIELD(wraparound_failsafe_count);
+}
+
+static void
+pgstat_accumulate_extvac_stats_relations(PgStat_VacuumRelationCounts * dst, PgStat_VacuumRelationCounts * src)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB && src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ ACCUMULATE_SUBFIELD(common, blks_fetched);
+ ACCUMULATE_SUBFIELD(common, blks_hit);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(table, pages_scanned);
+ ACCUMULATE_SUBFIELD(table, pages_removed);
+ ACCUMULATE_SUBFIELD(table, vm_new_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, tuples_frozen);
+ ACCUMULATE_SUBFIELD(table, recently_dead_tuples);
+ ACCUMULATE_SUBFIELD(table, index_vacuum_count);
+ ACCUMULATE_SUBFIELD(table, missed_dead_pages);
+ ACCUMULATE_SUBFIELD(table, missed_dead_tuples);
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(index, pages_deleted);
+ }
+}
+
+static void
+pgstat_accumulate_extvac_stats_db(PgStat_VacuumDBCounts * dst, PgStat_VacuumDBCounts * src)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+}
+
+/*
+ * Report that the table was just vacuumed and flush statistics.
+ */
+void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_VacuumRelation *shtabentry;
+ PgStatShared_VacuumDB *shdbentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_RELATION,
+ dboid, tableoid, false);
+ shtabentry = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+ pgstat_accumulate_extvac_stats_relations(&shtabentry->stats, params);
+
+ pgstat_unlock_entry(entry_ref);
+
+ if (!shared)
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_DB,
+ dboid, InvalidOid, false);
+ else
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_DB,
+ MyDatabaseId, InvalidOid, false);
+
+ shdbentry = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ pgstat_accumulate_common(&shdbentry->stats.common, ¶ms->common);
+
+ pgstat_unlock_entry(entry_ref);
+}
+
+/*
+ * Flush out pending stats for the entry
+ *
+ * If nowait is true, this function returns false if lock could not
+ * immediately acquired, otherwise true is returned.
+ */
+bool
+pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumRelation *shtabstats;
+ PgStat_RelationVacuumPending *pendingent; /* table entry of shared stats */
+
+ pendingent = (PgStat_RelationVacuumPending *) entry_ref->pending;
+ shtabstats = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts.
+ */
+ if (pg_memory_is_all_zeros(&pendingent,
+ sizeof(struct PgStat_RelationVacuumPending)))
+ return true;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ {
+ return false;
+ }
+
+ pgstat_accumulate_extvac_stats_relations(&(shtabstats->stats), &(pendingent->counts));
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the vacuum collected statistics for one relation or NULL.
+ */
+PgStat_VacuumRelationCounts *
+pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid)
+{
+ return (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_RELATION, dbid, relid);
+}
+
+PgStat_VacuumDBCounts *
+pgstat_fetch_stat_vacuum_dbentry(Oid dbid)
+{
+ return (PgStat_VacuumDBCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_DB, dbid, InvalidOid);
+}
+
+bool
+pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumDB *sharedent;
+ PgStat_VacuumDBCounts *pendingent;
+
+ pendingent = (PgStat_VacuumDBCounts *) entry_ref->pending;
+ sharedent = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ /* The entry was successfully flushed, add the same to database stats */
+ pgstat_accumulate_extvac_stats_db(&(sharedent->stats), pendingent);
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Find or create a local PgStat_VacuumDBCounts entry for dboid.
+ */
+PgStat_VacuumDBCounts *
+pgstat_prep_vacuum_database_pending(Oid dboid)
+{
+ PgStat_EntryRef *entry_ref;
+
+ /*
+ * This should not report stats on database objects before having
+ * connected to a database.
+ */
+ Assert(!OidIsValid(dboid) || OidIsValid(MyDatabaseId));
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_VACUUM_DB, dboid, InvalidOid,
+ NULL);
+
+ if (entry_ref == NULL)
+ return NULL;
+
+ return entry_ref->pending;
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 901f3dd55a1..38f16ec4c85 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2313,3 +2313,221 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
+
+ if (!pending)
+ {
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, 0);
+
+ if (!pending)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
+ }
+
+ extvacuum = pending;
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
+
+ if (!pending)
+ {
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, 0);
+
+ if (!pending)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
+ }
+
+ extvacuum = pending;
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_VacuumDBCounts *extvacuum;
+ PgStat_VacuumDBCounts *pending;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (!OidIsValid(dbid))
+ PG_RETURN_VOID();
+
+ pending = pgstat_fetch_stat_vacuum_dbentry(dbid);
+
+ if (!pending)
+ PG_RETURN_VOID();
+
+ extvacuum = pending;
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index f0260e6e412..dd4c4522355 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -3091,6 +3091,12 @@
boot_val => 'false',
},
+{ name => 'track_vacuum_statistics', type => 'bool', context => 'PGC_SUSET', group => 'STATS_CUMULATIVE',
+ short_desc => 'Collects vacuum statistics for vacuum activity.',
+ variable => 'pgstat_track_vacuum_statistics',
+ boot_val => 'false',
+},
+
{ name => 'track_wal_io_timing', type => 'bool', context => 'PGC_SUSET', group => 'STATS_CUMULATIVE',
short_desc => 'Collects timing statistics for WAL I/O activity.',
variable => 'track_wal_io_timing',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index c4f92fcdac8..b079ebead05 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -669,6 +669,7 @@
#track_wal_io_timing = off
#track_functions = none # none, pl, all
#stats_fetch_consistency = cache # cache, none, snapshot
+#track_vacuum_statistics = off
# - Monitoring -
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 961337ce282..93f162c2133 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12640,6 +12640,15 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int4,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
# oid8 related functions
{ oid => '8255', descr => 'convert oid to oid8',
proname => 'oid8', prorettype => 'oid8', proargtypes => 'oid',
@@ -12705,4 +12714,22 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,wraparound_failsafe,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index e885a4b9c77..bf4ddf74568 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -300,6 +301,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -332,6 +353,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
@@ -412,4 +434,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 072065adc90..9f1a1ee5c23 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -114,6 +114,15 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -161,6 +170,112 @@ typedef struct PgStat_TableCounts
PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
+typedef struct PgStat_CommonCounts
+{
+ /* blocks */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* heap blocks */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* WAL */
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+
+ /* Time */
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+
+ /* tuples */
+ int64 tuples_deleted;
+
+ /* failsafe */
+ int32 wraparound_failsafe_count;
+} PgStat_CommonCounts;
+
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are
+ * still visible to some
+ * transaction */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due
+ * to failure to get a cleanup
+ * lock */
+ int64 pages_scanned; /* heap pages examined (not skipped by
+ * VM) */
+ int64 pages_removed; /* heap pages removed by vacuum
+ * "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as
+ * all-visible */
+ int64 vm_new_frozen_pages; /* pages marked in VM as
+ * frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as
+ * all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as
+ * all-visible and
+ * frozen */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */ ;
+} PgStat_VacuumRelationCounts;
+
+typedef struct PgStat_VacuumRelationStatus
+{
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
+
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ PgStat_CommonCounts common;
+ int32 errors;
+} PgStat_VacuumDBCounts;
+
/* ----------
* PgStat_TableStatus Per-table status within a backend
*
@@ -185,6 +300,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -812,6 +933,16 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+ pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts * pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts * pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid);
+PgStat_VacuumDBCounts *pgstat_fetch_stat_vacuum_dbentry(Oid dbid);
+
/*
* Functions in pgstat_wal.c
*/
@@ -828,7 +959,8 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
-
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics_for_relations;
/*
* Variables in pgstat_bgwriter.c
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 9b8fbae00ed..3ffb8395396 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -500,6 +500,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -678,6 +690,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index c30b6235623..ded9767b677 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6d960423912
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 6a4d3532e03..6c3bce90d6d 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -100,6 +100,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..cfec3159580
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/recovery/t/052_vacuum_extending_basic_test.pl b/src/test/recovery/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..a5a0c195e67
--- /dev/null
+++ b/src/test/recovery/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,737 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • pg_stat_vacuum_tables
+# • pg_stat_vacuum_indexes
+# • pg_stat_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server logging level for the test
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+ track_vacuum_statistics = on
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls pg_stat_vacuum_tables and pg_stat_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM pg_stat_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, total_blks_dirtied,
+ total_blks_written, wal_records,
+ wal_fpi, wal_bytes
+ FROM pg_stat_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = pg_stat_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_* update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_* update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_* update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_* update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_* update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM pg_stat_get_vacuum_tables($reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM pg_stat_get_vacuum_indexes($reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM pg_stat_get_vacuum_tables(1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM pg_stat_get_vacuum_indexes(1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM pg_stat_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM pg_stat_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM pg_stat_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#------------------------------------------------------------------------------
+# Test 9: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM pg_stat_get_vacuum_tables(0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM pg_stat_get_vacuum_indexes(0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM pg_stat_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'pg_stat_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM pg_stat_vacuum_indexes WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'pg_stat_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM pg_stat_get_vacuum_tables($reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM pg_stat_get_vacuum_indexes($indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM pg_stat_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/src/test/recovery/t/053_vacuum_extending_freeze_test.pl b/src/test/recovery/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..82089c013f4
--- /dev/null
+++ b/src/test/recovery/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,329 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats system using TAP
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server for aggressive freezing behavior used by the test
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_vacuum_statistics = on
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Enable necessary settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+# wait_for_vacuum_stats
+#
+# Polls pg_stat_vacuum_tables until the named columns exceed the provided
+# baseline values or until timeout. Callers should pass:
+#
+# tab_frozen_column => 'vm_new_frozen_pages' # column name (string) or 'rev_all_frozen_pages'
+# tab_visible_column => 'vm_new_visible_pages' # column name (string) or 'rev_all_visible_pages'
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 or 1 # if true, run vacuum_sql before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_frozen_column = $args{tab_frozen_column};
+ my $tab_visible_column = $args{tab_visible_column};
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count};
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count};
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+ my $vacuum_run = 0;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ $vacuum_run = 1;
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_rev_all_frozen_pages(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_rev_all_visible_pages(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ # sub-second sleep
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM pg_stat_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+# Poll the stats view until the expected deltas appear or timeout.
+# We do not expect rev_all_* counters to change here, so we pass -1 for them.
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Verify initial statistics after vacuum
+#------------------------------------------------------------------------------
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+# Poll until stats update or timeout.
+# We do not expect vm_new_visible_frozen_pages or pages_all_visible to change here,
+# so we pass -1 for those counters.
+$updated = wait_for_vacuum_stats(
+ tab_frozen_column => 'rev_all_frozen_pages',
+ tab_visible_column => 'rev_all_visible_pages',
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Check updated statistics after backend activity
+#------------------------------------------------------------------------------
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+# Poll until stats update or timeout.
+# We pass current snapshot values for vm_new_visible_frozen_pages and expect rev counters unchanged.
+$updated = wait_for_vacuum_stats(
+ tab_frozen_column => 'vm_new_visible_frozen_pages',
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ run_vacuum => 1,
+ single_column => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Verify statistics after final vacuum
+# Check updated stats after backend work
+#------------------------------------------------------------------------------
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 7d26bc1a1dc..d4dc44970d2 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2332,6 +2332,81 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT d.oid AS dboid,
+ d.datname AS dbname,
+ s.db_blks_read,
+ s.db_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time,
+ s.wraparound_failsafe,
+ s.errors
+ FROM pg_database d,
+ LATERAL pg_stat_get_vacuum_database(d.oid) s(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
+pg_stat_vacuum_indexes| SELECT c.oid AS relid,
+ i.oid AS indexrelid,
+ n.nspname AS schemaname,
+ c.relname,
+ i.relname AS indexrelname,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_deleted,
+ s.tuples_deleted,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (((pg_class c
+ JOIN pg_index x ON ((c.oid = x.indrelid)))
+ JOIN pg_class i ON ((i.oid = x.indexrelid)))
+ LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(i.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
+pg_stat_vacuum_tables| SELECT n.nspname AS schemaname,
+ c.relname,
+ s.relid,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_scanned,
+ s.pages_removed,
+ s.vm_new_frozen_pages,
+ s.vm_new_visible_pages,
+ s.vm_new_visible_frozen_pages,
+ s.missed_dead_pages,
+ s.tuples_deleted,
+ s.tuples_frozen,
+ s.recently_dead_tuples,
+ s.missed_dead_tuples,
+ s.wraparound_failsafe,
+ s.index_vacuum_count,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (pg_class c
+ JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(c.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
--
2.39.5 (Apple Git-154)
From d38ba152b2271965b220a3e75bd3b0d17f6e1234 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 3/3] Add documentation about the system views that are used in
the machinery of vacuum statistics.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/system-views.sgml | 755 +++++++++++++++++++++++++++++++++
1 file changed, 755 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 8b4abef8c68..91b278596fd 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5683,4 +5683,759 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v27-0001-Introduce-new-statistics-tracking-the-number-of-time.patch (9.3K, 3-v27-0001-Introduce-new-statistics-tracking-the-number-of-time.patch)
download | inline diff:
From f1efce79f0aa61c650ec7cc6f40bf6a0806d29f5 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 18:30:12 +0300
Subject: [PATCH 1/3] Introduce new statistics tracking the number of times the
all-visible and all-frozen bits are cleared in the visibility map
(rev_all_visible_pages and rev_all_frozen_pages). These counters, together
with the existing per-vacuum frozen page statistics (vm_new_frozen_pages,
vm_new_visible_pages), help assess how aggressively vacuum is configured and
how frequently the backend has to revoke all-frozen/all-visible bits due to
concurrent modifications.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/visibilitymap.c | 10 ++++++++++
src/backend/catalog/system_views.sql | 4 +++-
src/backend/utils/activity/pgstat_relation.c | 2 ++
src/backend/utils/adt/pgstatfuncs.c | 6 ++++++
src/include/catalog/pg_proc.dat | 12 +++++++++++-
src/include/pgstat.h | 18 +++++++++++++++++-
src/test/regress/expected/rules.out | 12 +++++++++---
7 files changed, 58 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 3047bd46def..2e7c28ea307 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -161,6 +162,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 7553f31fef0..fa102f9c270 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -727,7 +727,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..885d590d2b2 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 73ca0bb0b7f..901f3dd55a1 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5e5e33f64fc..961337ce282 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12693,6 +12693,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index fff7ecc2533..072065adc90 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -156,6 +156,9 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -214,7 +217,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -463,6 +466,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autoanalyze_time;
TimestampTz stat_reset_time;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_StatTabEntry;
/* ------
@@ -722,6 +727,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f4ee2bd7459..7d26bc1a1dc 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2268,7 +2270,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2323,7 +2327,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
[text/plain] v27-0002-Add-machinery-for-grabbing-an-extended-vacuum-statis.patch (113.4K, 4-v27-0002-Add-machinery-for-grabbing-an-extended-vacuum-statis.patch)
download | inline diff:
From 20e1e8f82e980f31e6855820945067b7268a4962 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 19:55:30 +0300
Subject: [PATCH 2/3] Add machinery for grabbing an extended vacuum statistics.
Vacuum statistics are stored separately from regular relation and
database statistics. Dedicated PGSTAT_KIND_VACUUM_RELATION and
PGSTAT_KIND_VACUUM_DB entries in the cumulative statistics system
allocate memory for vacuum-specific metrics, reducing overall memory use.
Statistics are gathered separately for tables and indexes according to
vacuum phases. The ExtVacReport union and type field distinguish
PGSTAT_EXTVAC_TABLE vs PGSTAT_EXTVAC_INDEX. Heap vacuum stats are sent
to the cumulative statistics system after vacuum has processed the
indexes, according to vacuum phases. Database vacuum statistics
aggregate statistics of tables and indexes in the database.
Common for tables, indexes, and database:
total_blks_hit, total_blks_read, total_blks_dirtied are number of hit,
missed and dirtied pages in shared buffers during a vacuum operation
respectively. total_blks_dirtied means 'dirtied only by this action',
so if a page was dirty before the vacuum operation, it does not count
as dirtied.
blk_read_time and blk_write_time track only access to buffer pages and
flushing them to disk. During vacuum, write time can remain zero if no
flushing operations were performed. total_time is the diff between
timestamps at start and finish; it includes idle time (IO and lock
waits), so it is not equal to the sum of user and system time.
delay_time means total vacuum sleep time in vacuum delay point.
Table and index:
tuples_deleted is the number of tuples cleaned up by the vacuum
operation.
pages_removed is the number of pages by which the physical data storage
of the relation was reduced. pages_deleted is the number of freed pages
in the table (file size may not have changed).
They are processed as independent structures that do not affect each
other, unlike WAL and buffers, so they cannot be summed for the
database as the common statistics above.
Table only:
pages_frozen is the number of pages marked as frozen in VM during
vacuum; it is incremented when a page is marked all-frozen.
pages_all_visible is the number of pages marked as all-visible in VM.
wraparound_failsafe_count is the number of times vacuum started urgent
cleanup to prevent wraparound.
Table and database:
wraparound_failsafe is the count of urgent anti-wraparound cleanups.
Database only:
errors is the number of errors at error level caught during vacuum.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 316 +++++++-
src/backend/catalog/heap.c | 1 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/system_views.sql | 109 +++
src/backend/commands/dbcommands.c | 1 +
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 10 +
src/backend/utils/activity/Makefile | 1 +
src/backend/utils/activity/pgstat.c | 30 +-
src/backend/utils/activity/pgstat_database.c | 9 +
src/backend/utils/activity/pgstat_relation.c | 6 +
src/backend/utils/activity/pgstat_vacuum.c | 217 ++++++
src/backend/utils/adt/pgstatfuncs.c | 218 ++++++
src/backend/utils/misc/guc_parameters.dat | 6 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 27 +
src/include/commands/vacuum.h | 26 +
src/include/pgstat.h | 134 +++-
src/include/utils/pgstat_internal.h | 15 +
src/include/utils/pgstat_kind.h | 4 +-
.../vacuum-extending-in-repetable-read.out | 53 ++
src/test/isolation/isolation_schedule | 1 +
.../vacuum-extending-in-repetable-read.spec | 59 ++
.../t/052_vacuum_extending_basic_test.pl | 737 ++++++++++++++++++
.../t/053_vacuum_extending_freeze_test.pl | 329 ++++++++
src/test/regress/expected/rules.out | 75 ++
26 files changed, 2380 insertions(+), 10 deletions(-)
create mode 100644 src/backend/utils/activity/pgstat_vacuum.c
create mode 100644 src/test/isolation/expected/vacuum-extending-in-repetable-read.out
create mode 100644 src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 src/test/recovery/t/052_vacuum_extending_basic_test.pl
create mode 100644 src/test/recovery/t/053_vacuum_extending_freeze_test.pl
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 4be267ff657..60db695b8f0 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -280,6 +280,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -399,6 +401,12 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* number of emergency vacuums to
+ * prevent anti-wraparound
+ * shutdown */
+
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -410,7 +418,6 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
-
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
static void heap_vacuum_eager_scan_setup(LVRelState *vacrel,
@@ -484,6 +491,227 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ TimestampTz starttime;
+
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ starttime = GetCurrentTimestamp();
+
+ counters->starttime = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+
+ /*
+ * if something goes wrong or user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read += bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit += bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied += bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written += bufusage.shared_blks_written;
+
+ report->wal_records += walusage.wal_records;
+ report->wal_fpi += walusage.wal_fpi;
+ report->wal_bytes += walusage.wal_bytes;
+
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time += VacuumDelayTime - counters->VacuumDelayTime;
+
+ report->total_time += secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched +=
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit +=
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
+
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Set initial values for common heap and index statistics */
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+
+ extvac_stats_end(rel, &counters->common, &report->common);
+
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->common.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/* Accumulate vacuum statistics for heap.
+ *
+ * Because of complexity of vacuum processing: it switch procesing between
+ * the heap relation to index relations and visa versa, we need to store
+ * gathered statistics information for heap relations several times before
+ * the vacuum starts processing the indexes again.
+ *
+ * It is necessary to gather correct statistics information for heap and indexes
+ * otherwice the index statistics information would be added to his parent heap
+ * statistics information and it would be difficult to analyze it later.
+ *
+ * We can't subtract union vacuum statistics information for index from the heap relations
+ * because of total and delay time time statistics collecting during parallel vacuum
+ * procudure.
+*/
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ /* Fill heap-specific extended stats fields */
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->vm_new_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->vm_new_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->vm_new_visible_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+
+}
+
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ /* Fill heap-specific extended stats fields */
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
/*
@@ -635,7 +863,6 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
new_rel_allvisible,
new_rel_allfrozen;
PGRUsage ru0;
- TimestampTz starttime = 0;
PgStat_Counter startreadtime = 0,
startwritetime = 0;
WalUsage startwalusage = pgWalUsage;
@@ -643,6 +870,11 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ /* Initialize vacuum statistics */
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
@@ -657,8 +889,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
}
}
- /* Used for instrumentation and stats report */
- starttime = GetCurrentTimestamp();
+ extvac_stats_start(rel, &extVacCounters);
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
@@ -671,6 +902,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
pgstat_progress_update_param(PROGRESS_VACUUM_STARTED_BY,
PROGRESS_VACUUM_STARTED_BY_MANUAL);
+ extvac_stats_start(rel, &extVacCounters);
+
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -687,6 +920,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -695,6 +929,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
+
/* Set up high level stuff about rel and its indexes */
vacrel->rel = rel;
vac_open_indexes(vacrel->rel, RowExclusiveLock, &vacrel->nindexes,
@@ -797,6 +1034,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->aggressive = vacuum_get_cutoffs(rel, params, &vacrel->cutoffs);
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ vacrel->wraparound_failsafe_count = 0;
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
@@ -959,6 +1197,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ /* extvac_stats_end(rel, &extVacCounters, &extVacReport.common); */
+
/*
* Report results to the cumulative stats system, too.
*
@@ -969,11 +1210,19 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
+
+ /*
+ * Make generic extended vacuum stats report and fill heap-specific
+ * extended stats fields.
+ */
+ extvac_stats_end(vacrel->rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+ pgstat_report_vacuum_extstats(vacrel->reloid, rel->rd_rel->relisshared, &extVacReport);
pgstat_report_vacuum(rel,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
- starttime);
+ extVacCounters.starttime);
pgstat_progress_end_command();
if (instrument)
@@ -981,7 +1230,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
TimestampTz endtime = GetCurrentTimestamp();
if (verbose || params.log_vacuum_min_duration == 0 ||
- TimestampDifferenceExceeds(starttime, endtime,
+ TimestampDifferenceExceeds(extVacCounters.starttime, endtime,
params.log_vacuum_min_duration))
{
long secs_dur;
@@ -997,7 +1246,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
int64 total_blks_read;
int64 total_blks_dirtied;
- TimestampDifference(starttime, endtime, &secs_dur, &usecs_dur);
+ TimestampDifference(extVacCounters.starttime, endtime, &secs_dur, &usecs_dur);
memset(&walusage, 0, sizeof(WalUsage));
WalUsageAccumDiff(&walusage, &pgWalUsage, &startwalusage);
memset(&bufferusage, 0, sizeof(BufferUsage));
@@ -1940,6 +2189,7 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
/* Count the newly all-frozen pages for logging */
vacrel->vm_new_visible_pages++;
+ vacrel->vm_new_frozen_pages++;
vacrel->vm_new_visible_frozen_pages++;
}
@@ -2662,10 +2912,20 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
vacrel->num_index_scans);
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
* that parallel VACUUM only gets the precheck and this postcheck.
@@ -3012,6 +3272,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3094,10 +3355,20 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
}
else
{
+ LVExtStatCounters counters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
+
+ extvac_stats_start(vacrel->rel, &counters);
+
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
estimated_count);
+
+ extvac_stats_end(vacrel->rel, &counters, &extVacReport.common);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
}
/* Reset the progress counters */
@@ -3123,6 +3394,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ memset(&extVacReport, 0, sizeof(PgStat_VacuumRelationCounts));
+ memset(&extVacReport.common, 0, sizeof(PgStat_CommonCounts));
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3141,6 +3420,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3149,6 +3429,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_extstats(vacrel->indoid, indrel->rd_rel->relisshared, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3173,6 +3461,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3192,12 +3485,21 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ if (!ParallelVacuumIsActive(vacrel))
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 606434823cf..024946766d8 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1883,6 +1883,7 @@ heap_drop_with_catalog(Oid relid)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(rel);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(rel));
/*
* Close relcache entry, but *keep* AccessExclusiveLock on the relation
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 43de42ce39e..f006287cbf5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2325,6 +2325,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* ensure that stats are dropped if transaction commits */
pgstat_drop_relation(userIndexRelation);
+ pgstat_vacuum_relation_delete_pending_cb(RelationGetRelid(userIndexRelation));
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index fa102f9c270..c9d19c2fb18 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1454,3 +1454,112 @@ REVOKE ALL ON pg_aios FROM PUBLIC;
GRANT SELECT ON pg_aios TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_aios() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_aios() TO pg_read_all_stats;
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stat_vacuum_tables AS
+ SELECT
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ S.relid as relid,
+
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
+
+ S.pages_scanned AS pages_scanned,
+ S.pages_removed AS pages_removed,
+ S.vm_new_frozen_pages AS vm_new_frozen_pages,
+ S.vm_new_visible_pages AS vm_new_visible_pages,
+ S.vm_new_visible_frozen_pages AS vm_new_visible_frozen_pages,
+ S.missed_dead_pages AS missed_dead_pages,
+ S.tuples_deleted AS tuples_deleted,
+ S.tuples_frozen AS tuples_frozen,
+ S.recently_dead_tuples AS recently_dead_tuples,
+ S.missed_dead_tuples AS missed_dead_tuples,
+
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.index_vacuum_count AS index_vacuum_count,
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+
+ FROM pg_class C JOIN
+ pg_namespace N ON N.oid = C.relnamespace,
+ LATERAL pg_stat_get_vacuum_tables(C.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
+
+CREATE VIEW pg_stat_vacuum_indexes AS
+ SELECT
+ C.oid AS relid,
+ I.oid AS indexrelid,
+ N.nspname AS schemaname,
+ C.relname AS relname,
+ I.relname AS indexrelname,
+
+ S.total_blks_read AS total_blks_read,
+ S.total_blks_hit AS total_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.rel_blks_read AS rel_blks_read,
+ S.rel_blks_hit AS rel_blks_hit,
+
+ S.pages_deleted AS pages_deleted,
+ S.tuples_deleted AS tuples_deleted,
+
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time
+ FROM
+ pg_class C JOIN
+ pg_index X ON C.oid = X.indrelid JOIN
+ pg_class I ON I.oid = X.indexrelid
+ LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace),
+ LATERAL pg_stat_get_vacuum_indexes(I.oid) S
+ WHERE C.relkind IN ('r', 't', 'm');
+
+CREATE VIEW pg_stat_vacuum_database AS
+ SELECT
+ D.oid as dboid,
+ D.datname AS dbname,
+
+ S.db_blks_read AS db_blks_read,
+ S.db_blks_hit AS db_blks_hit,
+ S.total_blks_dirtied AS total_blks_dirtied,
+ S.total_blks_written AS total_blks_written,
+
+ S.wal_records AS wal_records,
+ S.wal_fpi AS wal_fpi,
+ S.wal_bytes AS wal_bytes,
+
+ S.blk_read_time AS blk_read_time,
+ S.blk_write_time AS blk_write_time,
+
+ S.delay_time AS delay_time,
+ S.total_time AS total_time,
+ S.wraparound_failsafe AS wraparound_failsafe,
+ S.errors AS errors
+ FROM
+ pg_database D,
+ LATERAL pg_stat_get_vacuum_database(D.oid) S;
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 87949054f26..9f10710636b 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -1815,6 +1815,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
* Tell the cumulative stats system to forget it immediately, too.
*/
pgstat_drop_database(db_id);
+ pgstat_drop_vacuum_database(db_id);
/*
* Except for the deletion of the catalog row, subsequent actions are not
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 03932f45c8a..769d0ba543f 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -117,6 +117,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2536,6 +2539,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index c3b3c9ea21a..16d215150cc 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -868,6 +868,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -876,6 +878,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ /* Set initial statistics values to gather vacuum statistics for the index */
+ extvac_stats_start_idx(indrel, &(indstats->istat), &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -904,6 +909,10 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_extstats(RelationGetRelid(indrel), indrel->rd_rel->relisshared, &extVacReport);
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1054,6 +1063,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile
index 0eb29ee78aa..df012ae3db5 100644
--- a/src/backend/utils/activity/Makefile
+++ b/src/backend/utils/activity/Makefile
@@ -27,6 +27,7 @@ OBJS = \
pgstat_function.o \
pgstat_io.o \
pgstat_relation.o \
+ pgstat_vacuum.o \
pgstat_replslot.o \
pgstat_shmem.o \
pgstat_slru.o \
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 11bb71cad5a..a854a64f135 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -203,7 +203,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind);
bool pgstat_track_counts = false;
int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE;
-
+bool pgstat_track_vacuum_statistics = false;
/* ----------
* state shared with pgstat_*.c
@@ -482,6 +482,34 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
},
+ [PGSTAT_KIND_VACUUM_DB] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+ /* so pg_stat_database entries can be seen in all databases */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumDB),
+ .shared_data_off = offsetof(PgStatShared_VacuumDB, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumDB *) 0)->stats),
+ .pending_size = sizeof(PgStat_VacuumDBCounts),
+
+ .flush_pending_cb = pgstat_vacuum_db_flush_cb,
+ },
+ [PGSTAT_KIND_VACUUM_RELATION] = {
+ .name = "vacuum statistics",
+
+ .fixed_amount = false,
+ .write_to_file = true,
+
+ .shared_size = sizeof(PgStatShared_VacuumRelation),
+ .shared_data_off = offsetof(PgStatShared_VacuumRelation, stats),
+ .shared_data_len = sizeof(((PgStatShared_VacuumRelation *) 0)->stats),
+ .pending_size = sizeof(PgStat_RelationVacuumPending),
+
+ .flush_pending_cb = pgstat_vacuum_relation_flush_cb
+ },
};
/*
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index d7f6d4c5ee6..079f02b3f03 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -46,6 +46,15 @@ pgstat_drop_database(Oid databaseid)
pgstat_drop_transactional(PGSTAT_KIND_DATABASE, databaseid, InvalidOid);
}
+/*
+ * Remove entry for the database being dropped.
+ */
+void
+pgstat_drop_vacuum_database(Oid databaseid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_DB, databaseid, InvalidOid);
+}
+
/*
* Called from autovacuum.c to report startup of an autovacuum process.
* We are called before InitPostgres is done, so can't rely on MyDatabaseId;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 885d590d2b2..9516552a43e 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -902,6 +902,12 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
return true;
}
+void
+pgstat_vacuum_relation_delete_pending_cb(Oid relid)
+{
+ pgstat_drop_transactional(PGSTAT_KIND_VACUUM_RELATION, relid, InvalidOid);
+}
+
void
pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref)
{
diff --git a/src/backend/utils/activity/pgstat_vacuum.c b/src/backend/utils/activity/pgstat_vacuum.c
new file mode 100644
index 00000000000..d3426e617a8
--- /dev/null
+++ b/src/backend/utils/activity/pgstat_vacuum.c
@@ -0,0 +1,217 @@
+#include "postgres.h"
+
+#include "pgstat.h"
+#include "utils/pgstat_internal.h"
+#include "utils/memutils.h"
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+bool pgstat_track_vacuum_statistics_for_relations = false;
+
+#define ACCUMULATE_FIELD(field) dst->field += src->field;
+
+#define ACCUMULATE_SUBFIELD(substruct, field) \
+ (dst->substruct.field += src->substruct.field)
+
+static void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUMULATE_FIELD(total_blks_read);
+ ACCUMULATE_FIELD(total_blks_hit);
+ ACCUMULATE_FIELD(total_blks_dirtied);
+ ACCUMULATE_FIELD(total_blks_written);
+
+ ACCUMULATE_FIELD(blks_fetched);
+ ACCUMULATE_FIELD(blks_hit);
+
+ ACCUMULATE_FIELD(wal_records);
+ ACCUMULATE_FIELD(wal_fpi);
+ ACCUMULATE_FIELD(wal_bytes);
+
+ ACCUMULATE_FIELD(blk_read_time);
+ ACCUMULATE_FIELD(blk_write_time);
+ ACCUMULATE_FIELD(delay_time);
+ ACCUMULATE_FIELD(total_time);
+
+ ACCUMULATE_FIELD(tuples_deleted);
+ ACCUMULATE_FIELD(wraparound_failsafe_count);
+}
+
+static void
+pgstat_accumulate_extvac_stats_relations(PgStat_VacuumRelationCounts * dst, PgStat_VacuumRelationCounts * src)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB && src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ ACCUMULATE_SUBFIELD(common, blks_fetched);
+ ACCUMULATE_SUBFIELD(common, blks_hit);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(table, pages_scanned);
+ ACCUMULATE_SUBFIELD(table, pages_removed);
+ ACCUMULATE_SUBFIELD(table, vm_new_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_pages);
+ ACCUMULATE_SUBFIELD(table, vm_new_visible_frozen_pages);
+ ACCUMULATE_SUBFIELD(table, tuples_frozen);
+ ACCUMULATE_SUBFIELD(table, recently_dead_tuples);
+ ACCUMULATE_SUBFIELD(table, index_vacuum_count);
+ ACCUMULATE_SUBFIELD(table, missed_dead_pages);
+ ACCUMULATE_SUBFIELD(table, missed_dead_tuples);
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ ACCUMULATE_SUBFIELD(common, tuples_deleted);
+ ACCUMULATE_SUBFIELD(index, pages_deleted);
+ }
+}
+
+static void
+pgstat_accumulate_extvac_stats_db(PgStat_VacuumDBCounts * dst, PgStat_VacuumDBCounts * src)
+{
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+}
+
+/*
+ * Report that the table was just vacuumed and flush statistics.
+ */
+void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_VacuumRelation *shtabentry;
+ PgStatShared_VacuumDB *shdbentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ if (!pgstat_track_vacuum_statistics)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_RELATION,
+ dboid, tableoid, false);
+ shtabentry = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+ pgstat_accumulate_extvac_stats_relations(&shtabentry->stats, params);
+
+ pgstat_unlock_entry(entry_ref);
+
+ if (!shared)
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_DB,
+ dboid, InvalidOid, false);
+ else
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_VACUUM_DB,
+ MyDatabaseId, InvalidOid, false);
+
+ shdbentry = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ pgstat_accumulate_common(&shdbentry->stats.common, ¶ms->common);
+
+ pgstat_unlock_entry(entry_ref);
+}
+
+/*
+ * Flush out pending stats for the entry
+ *
+ * If nowait is true, this function returns false if lock could not
+ * immediately acquired, otherwise true is returned.
+ */
+bool
+pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumRelation *shtabstats;
+ PgStat_RelationVacuumPending *pendingent; /* table entry of shared stats */
+
+ pendingent = (PgStat_RelationVacuumPending *) entry_ref->pending;
+ shtabstats = (PgStatShared_VacuumRelation *) entry_ref->shared_stats;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts.
+ */
+ if (pg_memory_is_all_zeros(&pendingent,
+ sizeof(struct PgStat_RelationVacuumPending)))
+ return true;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ {
+ return false;
+ }
+
+ pgstat_accumulate_extvac_stats_relations(&(shtabstats->stats), &(pendingent->counts));
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the vacuum collected statistics for one relation or NULL.
+ */
+PgStat_VacuumRelationCounts *
+pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid)
+{
+ return (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_RELATION, dbid, relid);
+}
+
+PgStat_VacuumDBCounts *
+pgstat_fetch_stat_vacuum_dbentry(Oid dbid)
+{
+ return (PgStat_VacuumDBCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_VACUUM_DB, dbid, InvalidOid);
+}
+
+bool
+pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStatShared_VacuumDB *sharedent;
+ PgStat_VacuumDBCounts *pendingent;
+
+ pendingent = (PgStat_VacuumDBCounts *) entry_ref->pending;
+ sharedent = (PgStatShared_VacuumDB *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ /* The entry was successfully flushed, add the same to database stats */
+ pgstat_accumulate_extvac_stats_db(&(sharedent->stats), pendingent);
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+/*
+ * Find or create a local PgStat_VacuumDBCounts entry for dboid.
+ */
+PgStat_VacuumDBCounts *
+pgstat_prep_vacuum_database_pending(Oid dboid)
+{
+ PgStat_EntryRef *entry_ref;
+
+ /*
+ * This should not report stats on database objects before having
+ * connected to a database.
+ */
+ Assert(!OidIsValid(dboid) || OidIsValid(MyDatabaseId));
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_VACUUM_DB, dboid, InvalidOid,
+ NULL);
+
+ if (entry_ref == NULL)
+ return NULL;
+
+ return entry_ref->pending;
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 901f3dd55a1..38f16ec4c85 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2313,3 +2313,221 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objid));
}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_TABLES_STATS_COLS 26
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_TABLES_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
+
+ if (!pending)
+ {
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, 0);
+
+ if (!pending)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
+ }
+
+ extvacuum = pending;
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->table.pages_scanned);
+ values[i++] = Int64GetDatum(extvacuum->table.pages_removed);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
+ values[i++] = Int64GetDatum(extvacuum->table.tuples_frozen);
+ values[i++] = Int64GetDatum(extvacuum->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(extvacuum->table.missed_dead_tuples);
+
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
+ values[i++] = Int64GetDatum(extvacuum->table.index_vacuum_count);
+
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_TABLES_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stat_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_INDEX_STATS_COLS 16
+
+ Oid relid = PG_GETARG_OID(0);
+ PgStat_VacuumRelationCounts *extvacuum;
+ PgStat_VacuumRelationCounts *pending;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_INDEX_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, MyDatabaseId);
+
+ if (!pending)
+ {
+ pending = pgstat_fetch_stat_vacuum_tabentry(relid, 0);
+
+ if (!pending)
+ {
+ InitMaterializedSRF(fcinfo, 0);
+ PG_RETURN_VOID();
+ }
+ }
+
+ extvacuum = pending;
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->common.blks_fetched -
+ extvacuum->common.blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.blks_hit);
+
+ values[i++] = Int64GetDatum(extvacuum->index.pages_deleted);
+ values[i++] = Int64GetDatum(extvacuum->common.tuples_deleted);
+
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+
+ Assert(i == PG_STAT_GET_VACUUM_INDEX_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
+Datum
+pg_stat_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_GET_VACUUM_DATABASE_STATS_COLS 14
+
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_VacuumDBCounts *extvacuum;
+ PgStat_VacuumDBCounts *pending;
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ bool nulls[PG_STAT_GET_VACUUM_DATABASE_STATS_COLS] = {0};
+ char buf[256];
+ int i = 0;
+
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (!OidIsValid(dbid))
+ PG_RETURN_VOID();
+
+ pending = pgstat_fetch_stat_vacuum_dbentry(dbid);
+
+ if (!pending)
+ PG_RETURN_VOID();
+
+ extvacuum = pending;
+
+ i = 0;
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_read);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_hit);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_dirtied);
+ values[i++] = Int64GetDatum(extvacuum->common.total_blks_written);
+
+ values[i++] = Int64GetDatum(extvacuum->common.wal_records);
+ values[i++] = Int64GetDatum(extvacuum->common.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, extvacuum->common.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(extvacuum->common.blk_read_time);
+ values[i++] = Float8GetDatum(extvacuum->common.blk_write_time);
+ values[i++] = Float8GetDatum(extvacuum->common.delay_time);
+ values[i++] = Float8GetDatum(extvacuum->common.total_time);
+ values[i++] = Int32GetDatum(extvacuum->common.wraparound_failsafe_count);
+ values[i++] = Int32GetDatum(extvacuum->errors);
+
+ Assert(i == PG_STAT_GET_VACUUM_DATABASE_STATS_COLS);
+
+ /* Returns the record as Datum */
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index f0260e6e412..dd4c4522355 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -3091,6 +3091,12 @@
boot_val => 'false',
},
+{ name => 'track_vacuum_statistics', type => 'bool', context => 'PGC_SUSET', group => 'STATS_CUMULATIVE',
+ short_desc => 'Collects vacuum statistics for vacuum activity.',
+ variable => 'pgstat_track_vacuum_statistics',
+ boot_val => 'false',
+},
+
{ name => 'track_wal_io_timing', type => 'bool', context => 'PGC_SUSET', group => 'STATS_CUMULATIVE',
short_desc => 'Collects timing statistics for WAL I/O activity.',
variable => 'track_wal_io_timing',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index c4f92fcdac8..b079ebead05 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -669,6 +669,7 @@
#track_wal_io_timing = off
#track_functions = none # none, pl, all
#stats_fetch_consistency = cache # cache, none, snapshot
+#track_vacuum_statistics = off
# - Monitoring -
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 961337ce282..93f162c2133 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12640,6 +12640,15 @@
proargnames => '{pid,io_id,io_generation,state,operation,off,length,target,handle_data_len,raw_result,result,target_desc,f_sync,f_localmem,f_buffered}',
prosrc => 'pg_get_aios' },
+{ oid => '8001',
+ descr => 'pg_stat_get_vacuum_tables returns vacuum stats values for table',
+ proname => 'pg_stat_get_vacuum_tables', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int4,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,vm_new_frozen_pages,vm_new_visible_pages,vm_new_visible_frozen_pages,missed_dead_pages,tuples_deleted,tuples_frozen,recently_dead_tuples,missed_dead_tuples,wraparound_failsafe,index_vacuum_count,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_tables' },
# oid8 related functions
{ oid => '8255', descr => 'convert oid to oid8',
proname => 'oid8', prorettype => 'oid8', proargtypes => 'oid',
@@ -12705,4 +12714,22 @@
proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_rev_all_frozen_pages' },
+{ oid => '8004',
+ descr => 'pg_stat_get_vacuum_indexes returns vacuum stats values for index',
+ proname => 'pg_stat_get_vacuum_indexes', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time}',
+ prosrc => 'pg_stat_get_vacuum_indexes' },
+{ oid => '8005',
+ descr => 'pg_stat_get_vacuum_database returns vacuum stats values for database',
+ proname => 'pg_stat_get_vacuum_database', prorows => 1000, provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,int4,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dbid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,total_time,wraparound_failsafe,errors}',
+ prosrc => 'pg_stat_get_vacuum_database' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index e885a4b9c77..bf4ddf74568 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -300,6 +301,26 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Counters and usage data for extended stats tracking.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -332,6 +353,7 @@ extern PGDLLIMPORT double vacuum_max_eager_freeze_failure_rate;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
@@ -412,4 +434,8 @@ extern double anl_random_fract(void);
extern double anl_init_selection_state(int n);
extern double anl_get_next_S(double t, int n, double *stateptr);
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report);
#endif /* VACUUM_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 072065adc90..9f1a1ee5c23 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -114,6 +114,15 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter conflict_count[CONFLICT_NUM_TYPES];
} PgStat_BackendSubEntry;
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -161,6 +170,112 @@ typedef struct PgStat_TableCounts
PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
+typedef struct PgStat_CommonCounts
+{
+ /* blocks */
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+
+ /* heap blocks */
+ int64 blks_fetched;
+ int64 blks_hit;
+
+ /* WAL */
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+
+ /* Time */
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+
+ /* tuples */
+ int64 tuples_deleted;
+
+ /* failsafe */
+ int32 wraparound_failsafe_count;
+} PgStat_CommonCounts;
+
+/* ----------
+ *
+ * PgStat_VacuumRelationCounts
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 recently_dead_tuples; /* deleted tuples that are
+ * still visible to some
+ * transaction */
+ int64 missed_dead_tuples; /* tuples not pruned by vacuum due
+ * to failure to get a cleanup
+ * lock */
+ int64 pages_scanned; /* heap pages examined (not skipped by
+ * VM) */
+ int64 pages_removed; /* heap pages removed by vacuum
+ * "truncation" */
+ int64 pages_frozen; /* pages marked in VM as frozen */
+ int64 pages_all_visible; /* pages marked in VM as
+ * all-visible */
+ int64 vm_new_frozen_pages; /* pages marked in VM as
+ * frozen */
+ int64 vm_new_visible_pages; /* pages marked in VM as
+ * all-visible */
+ int64 vm_new_visible_frozen_pages; /* pages marked in VM as
+ * all-visible and
+ * frozen */
+ int64 missed_dead_pages; /* pages with missed dead tuples */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } table;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ } index;
+ } /* per_type_stats */ ;
+} PgStat_VacuumRelationCounts;
+
+typedef struct PgStat_VacuumRelationStatus
+{
+ Oid id; /* table's OID */
+ bool shared; /* is it a shared catalog? */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_VacuumRelationStatus;
+
+typedef struct PgStat_VacuumDBCounts
+{
+ Oid dbjid;
+ PgStat_CommonCounts common;
+ int32 errors;
+} PgStat_VacuumDBCounts;
+
/* ----------
* PgStat_TableStatus Per-table status within a backend
*
@@ -185,6 +300,12 @@ typedef struct PgStat_TableStatus
Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
+typedef struct PgStat_RelationVacuumPending
+{
+ Oid id; /* table's OID */
+ PgStat_VacuumRelationCounts counts; /* event counts to be sent */
+} PgStat_RelationVacuumPending;
+
/* ----------
* PgStat_TableXactStatus Per-table, per-subtransaction status
* ----------
@@ -812,6 +933,16 @@ extern int pgstat_get_transactional_drops(bool isCommit, struct xl_xact_stats_it
extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_item *items, bool is_redo);
+extern void pgstat_drop_vacuum_database(Oid databaseid);
+extern void pgstat_vacuum_relation_delete_pending_cb(Oid relid);
+extern void
+ pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PgStat_RelationVacuumPending * find_vacuum_relation_entry(Oid relid);
+extern PgStat_VacuumDBCounts * pgstat_prep_vacuum_database_pending(Oid dboid);
+extern PgStat_VacuumRelationCounts * pgstat_fetch_stat_vacuum_tabentry(Oid relid, Oid dbid);
+PgStat_VacuumDBCounts *pgstat_fetch_stat_vacuum_dbentry(Oid dbid);
+
/*
* Functions in pgstat_wal.c
*/
@@ -828,7 +959,8 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_fetch_consistency;
-
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics;
+extern PGDLLIMPORT bool pgstat_track_vacuum_statistics_for_relations;
/*
* Variables in pgstat_bgwriter.c
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 9b8fbae00ed..3ffb8395396 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -500,6 +500,18 @@ typedef struct PgStatShared_Relation
PgStat_StatTabEntry stats;
} PgStatShared_Relation;
+typedef struct PgStatShared_VacuumDB
+{
+ PgStatShared_Common header;
+ PgStat_VacuumDBCounts stats;
+} PgStatShared_VacuumDB;
+
+typedef struct PgStatShared_VacuumRelation
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_VacuumRelation;
+
typedef struct PgStatShared_Function
{
PgStatShared_Common header;
@@ -678,6 +690,9 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+bool pgstat_vacuum_db_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+extern bool pgstat_vacuum_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
/*
* Functions in pgstat_archiver.c
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h
index c30b6235623..ded9767b677 100644
--- a/src/include/utils/pgstat_kind.h
+++ b/src/include/utils/pgstat_kind.h
@@ -38,9 +38,11 @@
#define PGSTAT_KIND_IO 10
#define PGSTAT_KIND_SLRU 11
#define PGSTAT_KIND_WAL 12
+#define PGSTAT_KIND_VACUUM_DB 13
+#define PGSTAT_KIND_VACUUM_RELATION 14
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
-#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
+#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VACUUM_RELATION
#define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1)
/* Custom stats kinds */
diff --git a/src/test/isolation/expected/vacuum-extending-in-repetable-read.out b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6d960423912
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,53 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 0| 0| 0| 0
+(1 row)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 600| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 300| 600| 0| 0| 303
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 6a4d3532e03..6c3bce90d6d 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -100,6 +100,7 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending-in-repetable-read
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..cfec3159580
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in pg_stat_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+ SET track_vacuum_statistics TO 'on';
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+ RESET track_vacuum_statistics;
+}
+
+session s1
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_vacuum_statistics TO 'on';
+ }
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM pg_stat_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/recovery/t/052_vacuum_extending_basic_test.pl b/src/test/recovery/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..a5a0c195e67
--- /dev/null
+++ b/src/test/recovery/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,737 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • pg_stat_vacuum_tables
+# • pg_stat_vacuum_indexes
+# • pg_stat_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server logging level for the test
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+ track_vacuum_statistics = on
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls pg_stat_vacuum_tables and pg_stat_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM pg_stat_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM pg_stat_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, total_blks_dirtied,
+ total_blks_written, wal_records,
+ wal_fpi, wal_bytes
+ FROM pg_stat_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = pg_stat_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_* update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_* update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_* update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_* update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_* update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM pg_stat_get_vacuum_tables($reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM pg_stat_get_vacuum_indexes($reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM pg_stat_get_vacuum_tables(1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM pg_stat_get_vacuum_indexes(1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM pg_stat_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM pg_stat_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM pg_stat_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#------------------------------------------------------------------------------
+# Test 9: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM pg_stat_get_vacuum_tables(0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM pg_stat_get_vacuum_indexes(0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM pg_stat_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'pg_stat_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM pg_stat_vacuum_indexes WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'pg_stat_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM pg_stat_get_vacuum_tables($reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM pg_stat_get_vacuum_indexes($indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM pg_stat_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/src/test/recovery/t/053_vacuum_extending_freeze_test.pl b/src/test/recovery/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..82089c013f4
--- /dev/null
+++ b/src/test/recovery/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,329 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats system using TAP
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server for aggressive freezing behavior used by the test
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_vacuum_statistics = on
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Enable necessary settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+# wait_for_vacuum_stats
+#
+# Polls pg_stat_vacuum_tables until the named columns exceed the provided
+# baseline values or until timeout. Callers should pass:
+#
+# tab_frozen_column => 'vm_new_frozen_pages' # column name (string) or 'rev_all_frozen_pages'
+# tab_visible_column => 'vm_new_visible_pages' # column name (string) or 'rev_all_visible_pages'
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 or 1 # if true, run vacuum_sql before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_frozen_column = $args{tab_frozen_column};
+ my $tab_visible_column = $args{tab_visible_column};
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count};
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count};
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+ my $vacuum_run = 0;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ $vacuum_run = 1;
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM pg_stat_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_rev_all_frozen_pages(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_rev_all_visible_pages(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ # sub-second sleep
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM pg_stat_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+# Poll the stats view until the expected deltas appear or timeout.
+# We do not expect rev_all_* counters to change here, so we pass -1 for them.
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Verify initial statistics after vacuum
+#------------------------------------------------------------------------------
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+# Poll until stats update or timeout.
+# We do not expect vm_new_visible_frozen_pages or pages_all_visible to change here,
+# so we pass -1 for those counters.
+$updated = wait_for_vacuum_stats(
+ tab_frozen_column => 'rev_all_frozen_pages',
+ tab_visible_column => 'rev_all_visible_pages',
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Check updated statistics after backend activity
+#------------------------------------------------------------------------------
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+# Poll until stats update or timeout.
+# We pass current snapshot values for vm_new_visible_frozen_pages and expect rev counters unchanged.
+$updated = wait_for_vacuum_stats(
+ tab_frozen_column => 'vm_new_visible_frozen_pages',
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ run_vacuum => 1,
+ single_column => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for pg_stat_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Verify statistics after final vacuum
+# Check updated stats after backend work
+#------------------------------------------------------------------------------
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 7d26bc1a1dc..d4dc44970d2 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2332,6 +2332,81 @@ pg_stat_user_tables| SELECT relid,
rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
+pg_stat_vacuum_database| SELECT d.oid AS dboid,
+ d.datname AS dbname,
+ s.db_blks_read,
+ s.db_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time,
+ s.wraparound_failsafe,
+ s.errors
+ FROM pg_database d,
+ LATERAL pg_stat_get_vacuum_database(d.oid) s(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time, wraparound_failsafe, errors);
+pg_stat_vacuum_indexes| SELECT c.oid AS relid,
+ i.oid AS indexrelid,
+ n.nspname AS schemaname,
+ c.relname,
+ i.relname AS indexrelname,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_deleted,
+ s.tuples_deleted,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (((pg_class c
+ JOIN pg_index x ON ((c.oid = x.indrelid)))
+ JOIN pg_class i ON ((i.oid = x.indexrelid)))
+ LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_indexes(i.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
+pg_stat_vacuum_tables| SELECT n.nspname AS schemaname,
+ c.relname,
+ s.relid,
+ s.total_blks_read,
+ s.total_blks_hit,
+ s.total_blks_dirtied,
+ s.total_blks_written,
+ s.rel_blks_read,
+ s.rel_blks_hit,
+ s.pages_scanned,
+ s.pages_removed,
+ s.vm_new_frozen_pages,
+ s.vm_new_visible_pages,
+ s.vm_new_visible_frozen_pages,
+ s.missed_dead_pages,
+ s.tuples_deleted,
+ s.tuples_frozen,
+ s.recently_dead_tuples,
+ s.missed_dead_tuples,
+ s.wraparound_failsafe,
+ s.index_vacuum_count,
+ s.wal_records,
+ s.wal_fpi,
+ s.wal_bytes,
+ s.blk_read_time,
+ s.blk_write_time,
+ s.delay_time,
+ s.total_time
+ FROM (pg_class c
+ JOIN pg_namespace n ON ((n.oid = c.relnamespace))),
+ LATERAL pg_stat_get_vacuum_tables(c.oid) s(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, vm_new_frozen_pages, vm_new_visible_pages, vm_new_visible_frozen_pages, missed_dead_pages, tuples_deleted, tuples_frozen, recently_dead_tuples, missed_dead_tuples, wraparound_failsafe, index_vacuum_count, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, total_time)
+ WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"]));
pg_stat_wal| SELECT wal_records,
wal_fpi,
wal_bytes,
--
2.39.5 (Apple Git-154)
[text/plain] v27-0003-Add-documentation-about-the-system-views-that-are-us.patch (25.1K, 5-v27-0003-Add-documentation-about-the-system-views-that-are-us.patch)
download | inline diff:
From d38ba152b2271965b220a3e75bd3b0d17f6e1234 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Thu, 19 Dec 2024 12:57:49 +0300
Subject: [PATCH 3/3] Add documentation about the system views that are used in
the machinery of vacuum statistics.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>, Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>, jian he <[email protected]>,
Kirill Reshke <[email protected]>, Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>, Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/system-views.sgml | 755 +++++++++++++++++++++++++++++++++
1 file changed, 755 insertions(+)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 8b4abef8c68..91b278596fd 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -5683,4 +5683,759 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</table>
</sect1>
+<sect1 id="view-pg-stat-vacuum-database">
+ <title><structname>pg_stat_vacuum_database</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-database">
+ <primary>pg_stat_vacuum_database</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_database</structname> will contain
+ one row for each database in the current cluster, showing statistics about
+ vacuuming that database.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_database</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this database
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this database, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this database, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this database, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>errors</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum operations performed on this database
+ were interrupted on any errors
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-indexes">
+ <title><structname>pg_stat_vacuum_indexes</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-indexes">
+ <primary>pg_stat_vacuum_indexes</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_indexes</structname> will contain
+ one row for each index in the current database (including TOAST
+ table indexes), showing statistics about vacuuming that specific index.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_indexes</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of an index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this index is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this index were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages deleted by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this index
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this index, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this index, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this index, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="view-pg-stat-vacuum-tables">
+ <title><structname>pg_stat_vacuum_tables</structname></title>
+
+ <indexterm zone="view-pg-stat-vacuum-tables">
+ <primary>pg_stat_vacuum_tables</primary>
+ </indexterm>
+
+ <para>
+ The view <structname>pg_stat_vacuum_tables</structname> will contain
+ one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table.
+ </para>
+
+ <table>
+ <title><structname>pg_stat_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the
+ buffer cache by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks written directly by vacuum or auto vacuum.
+ Blocks that are dirtied by a vacuum process can be written out by another process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this
+ table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were already found
+ in the buffer cache by vacuum operations, so that a read was not necessary
+ (this only includes hits in the
+ project; buffer cache, not the operating system's file system cache)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from the physical storage by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-frozen by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible by vacuum
+ in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of the number of pages newly set all-visible and all-frozen
+ by vacuum in the visibility map.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples of this table that vacuum operations marked as
+ frozen
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations left in this table due
+ to their visibility in transactions
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD (not just RECENTLY_DEAD) tuples that could not be
+ pruned due to failure to acquire a cleanup lock on a heap page.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times the vacuum was run to prevent a wraparound problem.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed_dead_tuples.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ performed on this table
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent reading database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>int8</type>
+ </para>
+ <para>
+ Time spent writing database blocks by vacuum operations performed on
+ this table, in milliseconds (if <xref linkend="guc-track-io-timing"/> is enabled,
+ otherwise zero)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent sleeping in a vacuum delay point by vacuum operations performed on
+ this table, in milliseconds (see <xref linkend="runtime-config-resource-vacuum-cost"/>
+ for details)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>system_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ System CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>user_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ User CPU time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Columns <structfield>total_*</structfield>, <structfield>wal_*</structfield>
+ and <structfield>blk_*</structfield> include data on vacuuming indexes on this table, while columns
+ <structfield>system_time</structfield> and <structfield>user_time</structfield> only include data
+ on vacuuming the heap.</para>
+ </sect1>
</chapter>
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-09 15:46 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2026-03-09 15:46 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
I discovered that my last patches were incorrectly formed. I updated the
correct version.
On 09.03.2026 18:25, Alena Rybakina wrote:
> I developed a patch with a different approach - using custom
> statistics. As in the previous approach, I store statistics separately
> in slots for relations and for databases. It was also necessary to
> introduce a hook, and all control is handled through an extension.
> Statistics collection in the core occurs only if the hook is defined
> (i.e., the extension is added to shared_preload_libraries).
> The extension is also controlled by additional gucs that allow
> disabling vacuum statistics collection, or collecting statistics only
> for system relations, only for user relations, or only at the database
> or relation level.
>
> For now, I have divided them into several categories: general
> statistics (including the number of removed tables and tuples, and how
> many times wraparound prevention occurred), cost-based statistics,
> buffer statistics, and timing statistics. Memory is dynamically freed
> or allocated when the corresponding guc configuration changes. This
> approach is still a work in progress.
>
> In the README and documentation in the extension, I also added
> information about how much memory will be used to store the objects
> (approximately 300 KB) and added the function to measure memory
> consumption.
>
> Currently there are three patches:
> The first patch still collects statistics about frozen pages and pages
> where the visibility flag is cleared.
> The second patch implements statistics collection in the core, but
> without storing them.
>
> The third patch is the extension itself, where the statistics are
> stored and displayed, with the control mechanisms described above.
>
>
> -----------
> Best regards,
> Alena Rybakina
From 7811d1b76b8e65c0eb364c8d113df7a304422a8a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 18:30:12 +0300
Subject: [PATCH 1/3] Introduce new statistics tracking the number of times the
all-visible and all-frozen bits are cleared in the visibility map
(rev_all_visible_pages and rev_all_frozen_pages). These counters, together
with the existing per-vacuum frozen page statistics (vm_new_frozen_pages,
vm_new_visible_pages), help assess how aggressively vacuum is configured and
how frequently the backend has to revoke all-frozen/all-visible bits due to
concurrent modifications.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/visibilitymap.c | 10 ++++++++++
src/backend/catalog/system_views.sql | 4 +++-
src/backend/utils/activity/pgstat_relation.c | 2 ++
src/backend/utils/adt/pgstatfuncs.c | 6 ++++++
src/include/catalog/pg_proc.dat | 12 +++++++++++-
src/include/pgstat.h | 18 +++++++++++++++++-
src/test/regress/expected/rules.out | 12 +++++++++---
7 files changed, 58 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..9ea7a068ef0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ecb7c996e86..1242eca7304 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -741,7 +741,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..885d590d2b2 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 50ea9e8fb83..83ff1fff87d 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..252eab079d6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12831,6 +12831,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..02fbb8480dd 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,9 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -466,6 +469,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autoanalyze_time;
TimestampTz stat_reset_time;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_StatTabEntry;
/* ------
@@ -725,6 +730,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index deb6e2ad6a9..e392428377d 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2279,7 +2281,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2334,7 +2338,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
From f78253895ef7e489e59389d60bc596b7cf42a19e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
Vacuum statistics are stored separately from regular relation and
database statistics. Dedicated PGSTAT_KIND_VACUUM_RELATION and
PGSTAT_KIND_VACUUM_DB entries in the cumulative statistics system
allocate memory for vacuum-specific metrics. Statistics are gathered
separately for tables and indexes according to vacuum phases. The
ExtVacReport union and type field distinguish PGSTAT_EXTVAC_TABLE vs
PGSTAT_EXTVAC_INDEX. Heap vacuum stats are sent to the cumulative
statistics system after vacuum has processed the indexes. Database
vacuum statistics aggregate per-table and per-index statistics within
the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 140 ++++++++++++++++++++++++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 8 ++
src/include/commands/vacuum.h | 28 ++++++
src/include/pgstat.h | 58 +++++++++++
5 files changed, 238 insertions(+)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 82c5b28e0ad..04b087e2a5c 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -282,6 +282,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -402,6 +404,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -488,6 +499,107 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -646,7 +758,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params.log_vacuum_min_duration >= 0));
@@ -663,6 +778,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -690,7 +807,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -801,6 +920,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -977,6 +1099,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
starttime);
+
pgstat_progress_end_command();
if (instrument)
@@ -3018,6 +3141,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3129,6 +3253,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3147,6 +3275,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3155,6 +3284,9 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3179,6 +3311,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3198,12 +3334,16 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 62c1ebdfd9b..faeab06d2bc 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2541,6 +2544,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 279108ca89f..7a85c644749 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,6 +869,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -877,6 +879,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -905,6 +909,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1055,6 +1062,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index e885a4b9c77..c50ce51e9da 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -333,6 +334,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 02fbb8480dd..7fe8e5468b8 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -92,6 +92,63 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -680,6 +737,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
From 7c42b68e7ebeeceb1475502d2ab5e2f9bd543670 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 3 Mar 2026 00:17:13 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics extension for extended vacuum
statistics.
This commit introduces the ext_vacuum_statistics extension, which provides
extended vacuum statistics through a dedicated schema and views. Statistics
are stored via the pgstat custom statistics infrastructure. The extension
registers set_report_vacuum_hook to receive vacuum metrics and persists
them into custom stats; when the hook is not set, no additional overhead
is incurred.
Views pg_stats_vacuum_tables, pg_stats_vacuum_indexes and
pg_stats_vacuum_database expose per-table, per-index and aggregated
per-database vacuum statistics respectively.
GUCs control which objects are tracked and how. vacuum_statistics.enabled
(default on) turns collection on or off. vacuum_statistics.object_types
(default all) restricts tracking to databases only, relations only, or both.
When tracking relations, vacuum_statistics.track_relations (default all)
filters by system or user tables. vacuum_statistics.track_databases_from_list
and vacuum_statistics.track_relations_from_list (both default off) restrict
tracking to databases and relations explicitly added via add_track_database
and add_track_relation; when off, all objects of the chosen types are tracked.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 +++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 260 +++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 +++++
.../t/054_vacuum_extending_gucs_test.pl | 203 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1000 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 +++++++++
doc/src/sgml/filelist.sgml | 1 +
src/backend/access/heap/vacuumlazy.c | 112 +-
src/backend/commands/vacuumparallel.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 24 +
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 11 +
23 files changed, 3576 insertions(+), 18 deletions(-)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 2f0a88d3f77..6e064c566aa 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..4f0b1877f90
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,260 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT VOLATILE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT VOLATILE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT VOLATILE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..0ba52f7988f
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_rev_all_frozen_pages(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_rev_all_visible_pages(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..b8d5bf30ecf
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,203 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra));
+ $node->safe_psql($db, $sql);
+ sleep(0.1);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no stats when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..1f6f3e90614
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1000 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+
+/* Hash tables for track_databases and track_relations_list */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+static void evs_track_load_file(void);
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+#define ACCUM_IF(dst, src, field) \
+ do { (dst)->field += (src)->field; } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read);
+ ACCUM_IF(dst, src, total_blks_hit);
+ ACCUM_IF(dst, src, total_blks_dirtied);
+ ACCUM_IF(dst, src, total_blks_written);
+ ACCUM_IF(dst, src, blks_fetched);
+ ACCUM_IF(dst, src, blks_hit);
+ ACCUM_IF(dst, src, blk_read_time);
+ ACCUM_IF(dst, src, blk_write_time);
+ ACCUM_IF(dst, src, delay_time);
+ ACCUM_IF(dst, src, total_time);
+ ACCUM_IF(dst, src, wal_records);
+ ACCUM_IF(dst, src, wal_fpi);
+ ACCUM_IF(dst, src, wal_bytes);
+ ACCUM_IF(dst, src, wraparound_failsafe_count);
+ ACCUM_IF(dst, src, interrupts_count);
+ ACCUM_IF(dst, src, tuples_deleted);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/* GUC assign hooks: parse string and update bit flags */
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ if (strcmp(newval, "databases") == 0)
+ evs_track_bits = EVS_TRACK_DATABASES;
+ else if (strcmp(newval, "relations") == 0)
+ evs_track_bits = EVS_TRACK_RELATIONS;
+ else
+ evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES; /* "all" or unknown */
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ if (strcmp(newval, "system") == 0)
+ evs_track_relations_bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(newval, "user") == 0)
+ evs_track_relations_bits = EVS_FILTER_USER;
+ else
+ evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER; /* "all" or unknown */
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0, NULL, evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0, NULL, evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ /* Hash of database OIDs to track specific databases */
+ evs_track_databases_hash = hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ /* Hash of (dboid, reloid) to track specific relations */
+ evs_track_relations_hash = hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+}
+
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[256];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' || !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ return;
+
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else
+ {
+ if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+ }
+ FreeFile(fp);
+}
+
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!DataDir || DataDir[0] == '\0' || !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s/%s.tmp", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(tmppath, "w");
+ if (!fp)
+ return;
+
+ fprintf(fp, "[databases]\n");
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ fprintf(fp, "%u\n", *entry);
+
+ fprintf(fp, "[relations]\n");
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ if (OidIsValid(rel_entry->dboid))
+ fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ fprintf(fp, "0 %u\n", rel_entry->reloid);
+ }
+
+ if (FreeFile(fp) != 0 || rename(tmppath, path) != 0)
+ unlink(tmppath);
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ {
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(!found); /* true if newly added */
+ }
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid, EXTVAC_OBJID(relid, type));
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid, EXTVAC_OBJID(relid, type));
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid, InvalidOid);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index def13257cbe..b8cb62d22f1 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 24b706b29ad..f8a5781bde3 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index ac66fcbdb57..b03257c6973 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 04b087e2a5c..d20d5dddcc0 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -601,6 +601,65 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
}
}
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
* Initializes the eager scan management related members of the LVRelState.
@@ -778,7 +837,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
- extvac_stats_start(rel, &extVacCounters);
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
@@ -1094,11 +1154,25 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
pgstat_progress_end_command();
@@ -3256,8 +3330,8 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
LVExtStatCountersIdx extVacCounters;
PgStat_VacuumRelationCounts extVacReport;
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3284,8 +3358,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -3314,8 +3393,8 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
LVExtStatCountersIdx extVacCounters;
PgStat_VacuumRelationCounts extVacReport;
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3341,8 +3420,13 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_cleanup_one_index(&ivinfo, istat);
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 7a85c644749..d0426e228b4 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -879,8 +879,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -909,8 +909,12 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 885d590d2b2..f0db10803d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -271,6 +271,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index c50ce51e9da..09f7775b85e 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -23,6 +23,7 @@
#include "catalog/pg_type.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
+#include "executor/instrument.h"
#include "storage/lock.h"
#include "utils/relcache.h"
#include "pgstat.h"
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7fe8e5468b8..1013a52de6e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,7 @@ typedef struct PgStat_FunctionCounts
* Working state needed to accumulate per-function-call timing statistics.
*/
/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
* Type of entry: table (heap), index, or database aggregate.
*/
typedef enum ExtVacReportType
@@ -738,6 +739,16 @@ extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v29-0001-Introduce-new-statistics-tracking-the-number-of-time.patch (9.3K, 2-v29-0001-Introduce-new-statistics-tracking-the-number-of-time.patch)
download | inline diff:
From 7811d1b76b8e65c0eb364c8d113df7a304422a8a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 18:30:12 +0300
Subject: [PATCH 1/3] Introduce new statistics tracking the number of times the
all-visible and all-frozen bits are cleared in the visibility map
(rev_all_visible_pages and rev_all_frozen_pages). These counters, together
with the existing per-vacuum frozen page statistics (vm_new_frozen_pages,
vm_new_visible_pages), help assess how aggressively vacuum is configured and
how frequently the backend has to revoke all-frozen/all-visible bits due to
concurrent modifications.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/visibilitymap.c | 10 ++++++++++
src/backend/catalog/system_views.sql | 4 +++-
src/backend/utils/activity/pgstat_relation.c | 2 ++
src/backend/utils/adt/pgstatfuncs.c | 6 ++++++
src/include/catalog/pg_proc.dat | 12 +++++++++++-
src/include/pgstat.h | 18 +++++++++++++++++-
src/test/regress/expected/rules.out | 12 +++++++++---
7 files changed, 58 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..9ea7a068ef0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ecb7c996e86..1242eca7304 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -741,7 +741,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..885d590d2b2 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 50ea9e8fb83..83ff1fff87d 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..252eab079d6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12831,6 +12831,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..02fbb8480dd 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,9 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -466,6 +469,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter total_autoanalyze_time;
TimestampTz stat_reset_time;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_StatTabEntry;
/* ------
@@ -725,6 +730,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index deb6e2ad6a9..e392428377d 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2279,7 +2281,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2334,7 +2338,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ rev_all_frozen_pages,
+ rev_all_visible_pages
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
[text/plain] v29-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch (17.9K, 3-v29-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch)
download | inline diff:
From f78253895ef7e489e59389d60bc596b7cf42a19e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
Vacuum statistics are stored separately from regular relation and
database statistics. Dedicated PGSTAT_KIND_VACUUM_RELATION and
PGSTAT_KIND_VACUUM_DB entries in the cumulative statistics system
allocate memory for vacuum-specific metrics. Statistics are gathered
separately for tables and indexes according to vacuum phases. The
ExtVacReport union and type field distinguish PGSTAT_EXTVAC_TABLE vs
PGSTAT_EXTVAC_INDEX. Heap vacuum stats are sent to the cumulative
statistics system after vacuum has processed the indexes. Database
vacuum statistics aggregate per-table and per-index statistics within
the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 140 ++++++++++++++++++++++++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 8 ++
src/include/commands/vacuum.h | 28 ++++++
src/include/pgstat.h | 58 +++++++++++
5 files changed, 238 insertions(+)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 82c5b28e0ad..04b087e2a5c 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -282,6 +282,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -402,6 +404,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -488,6 +499,107 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -646,7 +758,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params.options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params.log_vacuum_min_duration >= 0));
@@ -663,6 +778,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -690,7 +807,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -801,6 +920,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -977,6 +1099,7 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->recently_dead_tuples +
vacrel->missed_dead_tuples,
starttime);
+
pgstat_progress_end_command();
if (instrument)
@@ -3018,6 +3141,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3129,6 +3253,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3147,6 +3275,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3155,6 +3284,9 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3179,6 +3311,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -3198,12 +3334,16 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 62c1ebdfd9b..faeab06d2bc 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2541,6 +2544,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 279108ca89f..7a85c644749 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -869,6 +869,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -877,6 +879,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
+
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -905,6 +909,9 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1055,6 +1062,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Set cost-based vacuum delay */
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index e885a4b9c77..c50ce51e9da 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -25,6 +25,7 @@
#include "storage/buf.h"
#include "storage/lock.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -333,6 +334,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 02fbb8480dd..7fe8e5468b8 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -92,6 +92,63 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -680,6 +737,7 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
[text/plain] v29-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch (143.0K, 4-v29-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch)
download | inline diff:
From 7c42b68e7ebeeceb1475502d2ab5e2f9bd543670 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 3 Mar 2026 00:17:13 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics extension for extended vacuum
statistics.
This commit introduces the ext_vacuum_statistics extension, which provides
extended vacuum statistics through a dedicated schema and views. Statistics
are stored via the pgstat custom statistics infrastructure. The extension
registers set_report_vacuum_hook to receive vacuum metrics and persists
them into custom stats; when the hook is not set, no additional overhead
is incurred.
Views pg_stats_vacuum_tables, pg_stats_vacuum_indexes and
pg_stats_vacuum_database expose per-table, per-index and aggregated
per-database vacuum statistics respectively.
GUCs control which objects are tracked and how. vacuum_statistics.enabled
(default on) turns collection on or off. vacuum_statistics.object_types
(default all) restricts tracking to databases only, relations only, or both.
When tracking relations, vacuum_statistics.track_relations (default all)
filters by system or user tables. vacuum_statistics.track_databases_from_list
and vacuum_statistics.track_relations_from_list (both default off) restrict
tracking to databases and relations explicitly added via add_track_database
and add_track_relation; when off, all objects of the chosen types are tracked.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 +++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 260 +++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 +++++
.../t/054_vacuum_extending_gucs_test.pl | 203 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1000 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 +++++++++
doc/src/sgml/filelist.sgml | 1 +
src/backend/access/heap/vacuumlazy.c | 112 +-
src/backend/commands/vacuumparallel.c | 12 +-
src/backend/utils/activity/pgstat_relation.c | 24 +
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 11 +
23 files changed, 3576 insertions(+), 18 deletions(-)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 2f0a88d3f77..6e064c566aa 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..4f0b1877f90
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,260 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT VOLATILE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT VOLATILE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT VOLATILE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..0ba52f7988f
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_rev_all_frozen_pages(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_rev_all_visible_pages(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_frozen_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_rev_all_visible_pages(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..b8d5bf30ecf
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,203 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra));
+ $node->safe_psql($db, $sql);
+ sleep(0.1);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no stats when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..1f6f3e90614
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1000 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+
+/* Hash tables for track_databases and track_relations_list */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+static void evs_track_load_file(void);
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+#define ACCUM_IF(dst, src, field) \
+ do { (dst)->field += (src)->field; } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read);
+ ACCUM_IF(dst, src, total_blks_hit);
+ ACCUM_IF(dst, src, total_blks_dirtied);
+ ACCUM_IF(dst, src, total_blks_written);
+ ACCUM_IF(dst, src, blks_fetched);
+ ACCUM_IF(dst, src, blks_hit);
+ ACCUM_IF(dst, src, blk_read_time);
+ ACCUM_IF(dst, src, blk_write_time);
+ ACCUM_IF(dst, src, delay_time);
+ ACCUM_IF(dst, src, total_time);
+ ACCUM_IF(dst, src, wal_records);
+ ACCUM_IF(dst, src, wal_fpi);
+ ACCUM_IF(dst, src, wal_bytes);
+ ACCUM_IF(dst, src, wraparound_failsafe_count);
+ ACCUM_IF(dst, src, interrupts_count);
+ ACCUM_IF(dst, src, tuples_deleted);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/* GUC assign hooks: parse string and update bit flags */
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ if (strcmp(newval, "databases") == 0)
+ evs_track_bits = EVS_TRACK_DATABASES;
+ else if (strcmp(newval, "relations") == 0)
+ evs_track_bits = EVS_TRACK_RELATIONS;
+ else
+ evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES; /* "all" or unknown */
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ if (strcmp(newval, "system") == 0)
+ evs_track_relations_bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(newval, "user") == 0)
+ evs_track_relations_bits = EVS_FILTER_USER;
+ else
+ evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER; /* "all" or unknown */
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0, NULL, evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0, NULL, evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ /* Hash of database OIDs to track specific databases */
+ evs_track_databases_hash = hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ /* Hash of (dboid, reloid) to track specific relations */
+ evs_track_relations_hash = hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+}
+
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[256];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' || !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ return;
+
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else
+ {
+ if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+ }
+ FreeFile(fp);
+}
+
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!DataDir || DataDir[0] == '\0' || !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s/%s.tmp", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(tmppath, "w");
+ if (!fp)
+ return;
+
+ fprintf(fp, "[databases]\n");
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ fprintf(fp, "%u\n", *entry);
+
+ fprintf(fp, "[relations]\n");
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ if (OidIsValid(rel_entry->dboid))
+ fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ fprintf(fp, "0 %u\n", rel_entry->reloid);
+ }
+
+ if (FreeFile(fp) != 0 || rename(tmppath, path) != 0)
+ unlink(tmppath);
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ {
+ bool found;
+
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(!found); /* true if newly added */
+ }
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid, EXTVAC_OBJID(relid, type));
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid, EXTVAC_OBJID(relid, type));
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid, InvalidOid);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index def13257cbe..b8cb62d22f1 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 24b706b29ad..f8a5781bde3 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index ac66fcbdb57..b03257c6973 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 04b087e2a5c..d20d5dddcc0 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -601,6 +601,65 @@ extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
}
}
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
+
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
* Initializes the eager scan management related members of the LVRelState.
@@ -778,7 +837,8 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
- extvac_stats_start(rel, &extVacCounters);
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
@@ -1094,11 +1154,25 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
pgstat_progress_end_command();
@@ -3256,8 +3330,8 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
LVExtStatCountersIdx extVacCounters;
PgStat_VacuumRelationCounts extVacReport;
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3284,8 +3358,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
@@ -3314,8 +3393,8 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
LVExtStatCountersIdx extVacCounters;
PgStat_VacuumRelationCounts extVacReport;
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3341,8 +3420,13 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_cleanup_one_index(&ivinfo, istat);
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 7a85c644749..d0426e228b4 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -879,8 +879,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
- extvac_stats_start_idx(indrel, istat, &extVacCounters);
-
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -909,8 +909,12 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
- memset(&extVacReport, 0, sizeof(extVacReport));
- extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 885d590d2b2..f0db10803d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -271,6 +271,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index c50ce51e9da..09f7775b85e 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -23,6 +23,7 @@
#include "catalog/pg_type.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
+#include "executor/instrument.h"
#include "storage/lock.h"
#include "utils/relcache.h"
#include "pgstat.h"
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7fe8e5468b8..1013a52de6e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,7 @@ typedef struct PgStat_FunctionCounts
* Working state needed to accumulate per-function-call timing statistics.
*/
/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
* Type of entry: table (heap), index, or database aggregate.
*/
typedef enum ExtVacReportType
@@ -738,6 +739,16 @@ extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-12 12:02 Andrei Lepikhov <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Andrei Lepikhov @ 2026-03-12 12:02 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 9/3/26 16:46, Alena Rybakina wrote:
> I discovered that my last patches were incorrectly formed. I updated the
> correct version.
I see that v29-0001-* is a quite separate feature itself at the moment.
It makes sense to remove the commit message phrase for
vm_new_frozen_pages and vm_new_visible_pages, introduced in later patches.
This patch itself looks good to me.
--
regards, Andrei Lepikhov,
pgEdge
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-12 13:28 Andrei Lepikhov <[email protected]>
parent: Andrei Lepikhov <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Andrei Lepikhov @ 2026-03-12 13:28 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 12/3/26 13:02, Andrei Lepikhov wrote:
> On 9/3/26 16:46, Alena Rybakina wrote:
>> I discovered that my last patches were incorrectly formed. I updated
>> the correct version.
>
> I see that v29-0001-* is a quite separate feature itself at the moment.
> It makes sense to remove the commit message phrase for
> vm_new_frozen_pages and vm_new_visible_pages, introduced in later patches.
> This patch itself looks good to me.
Since this patch is almost ready for commit, I reviewed it carefully. I
noticed a documentation entry was missing, so I added it. Please see the
attachment.
While updating the patch file, I also made a few small adjustments,
including changing the parameter order in the struct and VIEW. The
commit message is also fixed.
In addition, it makes sense to discuss how these parameters are supposed
to be used. I see the following use cases:
1. Which tables have the most VM churn? - monitoring
rev_all_visible_pages normalised on the table size and its average tuple
width might expose the most suspicious tables (in terms of table
statistics).
2. DML Skew. Dividing rev_all_visible_pages by the number of tuple
updates/deletes, normalised by the average table and tuple sizes, might
indicate whether changes are localised within the table.
3. IndexOnlyScan effectiveness. Considering the speed of
rev_all_visible_pages change, normalised to the value of the
relallvisible statistic, we may detect tables where Index-Only Scan
might be inefficiently used.
Feel free to criticise it or add your own - I’m just a developer, not a
DBA. Also, I’m not sure what use cases there are for the
rev_all_frozen_pages parameter.
--
regards, Andrei Lepikhov,
pgEdge
From 96789144424e991aab44e7c8dfad9db4a2e368e1 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 18:30:12 +0300
Subject: [PATCH v30] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 ++++++++++++++++++++
src/backend/access/heap/visibilitymap.c | 10 ++++++
src/backend/catalog/system_views.sql | 2 ++
src/backend/utils/activity/pgstat_relation.c | 2 ++
src/backend/utils/adt/pgstatfuncs.c | 6 ++++
src/include/catalog/pg_proc.dat | 12 +++++++-
src/include/pgstat.h | 17 ++++++++++-
src/test/regress/expected/rules.out | 6 ++++
8 files changed, 85 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index cc014564c97..8ce0d0dd2cb 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4249,6 +4249,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-visible bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-visible, for example during an <command>INSERT</command>,
+ <command>UPDATE</command>, or <command>DELETE</command>.
+ A high rate of change in this counter means that index-only scans
+ on this table may frequently need to fall back to heap fetches,
+ and that vacuum must re-do visibility map work on those pages.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-frozen bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-frozen. A high value compared to the number of vacuum cycles
+ indicates that DML activity is frequently undoing the freezing work
+ performed by vacuum.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..9ea7a068ef0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 339c016e510..1eaf79fdb4e 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -729,6 +729,8 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_dead_tuples(C.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(C.oid) as last_vacuum,
pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum,
pg_stat_get_last_analyze_time(C.oid) as last_analyze,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..bb26e97898d 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5ac022274a7..6d7c4cc1ed2 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..252eab079d6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12831,6 +12831,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..849eea24f29 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +219,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -450,6 +452,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -725,6 +729,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f373ad704b6..4fb3167e99c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1834,6 +1834,8 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_dead_tuples(c.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum,
pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum,
pg_stat_get_last_analyze_time(c.oid) AS last_analyze,
@@ -2285,6 +2287,8 @@ pg_stat_sys_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
@@ -2340,6 +2344,8 @@ pg_stat_user_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
--
2.53.0
Attachments:
[text/plain] v30-0001-Track-table-VM-stability.patch (11.0K, 2-v30-0001-Track-table-VM-stability.patch)
download | inline diff:
From 96789144424e991aab44e7c8dfad9db4a2e368e1 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Sat, 28 Feb 2026 18:30:12 +0300
Subject: [PATCH v30] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 ++++++++++++++++++++
src/backend/access/heap/visibilitymap.c | 10 ++++++
src/backend/catalog/system_views.sql | 2 ++
src/backend/utils/activity/pgstat_relation.c | 2 ++
src/backend/utils/adt/pgstatfuncs.c | 6 ++++
src/include/catalog/pg_proc.dat | 12 +++++++-
src/include/pgstat.h | 17 ++++++++++-
src/test/regress/expected/rules.out | 6 ++++
8 files changed, 85 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index cc014564c97..8ce0d0dd2cb 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4249,6 +4249,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-visible bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-visible, for example during an <command>INSERT</command>,
+ <command>UPDATE</command>, or <command>DELETE</command>.
+ A high rate of change in this counter means that index-only scans
+ on this table may frequently need to fall back to heap fetches,
+ and that vacuum must re-do visibility map work on those pages.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-frozen bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-frozen. A high value compared to the number of vacuum cycles
+ indicates that DML activity is frequently undoing the freezing work
+ performed by vacuum.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..9ea7a068ef0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 339c016e510..1eaf79fdb4e 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -729,6 +729,8 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_dead_tuples(C.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(C.oid) as last_vacuum,
pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum,
pg_stat_get_last_analyze_time(C.oid) as last_analyze,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..bb26e97898d 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5ac022274a7..6d7c4cc1ed2 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..252eab079d6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12831,6 +12831,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..849eea24f29 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +219,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -450,6 +452,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -725,6 +729,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f373ad704b6..4fb3167e99c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1834,6 +1834,8 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_dead_tuples(c.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum,
pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum,
pg_stat_get_last_analyze_time(c.oid) AS last_analyze,
@@ -2285,6 +2287,8 @@ pg_stat_sys_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
@@ -2340,6 +2344,8 @@ pg_stat_user_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
--
2.53.0
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-12 18:10 Alena Rybakina <[email protected]>
parent: Andrei Lepikhov <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2026-03-12 18:10 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 12.03.2026 18:28, Andrei Lepikhov wrote:
>
> In addition, it makes sense to discuss how these parameters are
> supposed to be used. I see the following use cases:
>
> 1. Which tables have the most VM churn? - monitoring
> rev_all_visible_pages normalised on the table size and its average
> tuple width might expose the most suspicious tables (in terms of table
> statistics).
> 2. DML Skew. Dividing rev_all_visible_pages by the number of tuple
> updates/deletes, normalised by the average table and tuple sizes,
> might indicate whether changes are localised within the table.
> 3. IndexOnlyScan effectiveness. Considering the speed of
> rev_all_visible_pages change, normalised to the value of the
> relallvisible statistic, we may detect tables where Index-Only Scan
> might be inefficiently used.
>
>
I agree with all these points and I think we can add it in the
documentation.
On 12.03.2026 17:02, Andrei Lepikhov wrote:
> On 9/3/26 16:46, Alena Rybakina wrote:
>> I discovered that my last patches were incorrectly formed. I updated
>> the correct version.
>
> I see that v29-0001-* is a quite separate feature itself at the
> moment. It makes sense to remove the commit message phrase for
> vm_new_frozen_pages and vm_new_visible_pages, introduced in later
> patches.
> This patch itself looks good to me.
BTW, I have noticed that my third patch (from 29th - when I have added
ext_vacuum_statistics) is huge but I have no idea how to split it
logically. I'm not sure that separation by objects can simplify the
review process. Maybe I should add only base logic for the extension and
then gucs, what do you think?
Any suggestions are welcome here.
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-13 13:04 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 2 replies; 46+ messages in thread
From: Alena Rybakina @ 2026-03-13 13:04 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; pgsql-hackers; +Cc: Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 13.03.2026 15:51, Alena Rybakina wrote:
>>>
>>> In addition, it makes sense to discuss how these parameters are
>>> supposed to be used. I see the following use cases:
>>>
>>> 1. Which tables have the most VM churn? - monitoring
>>> rev_all_visible_pages normalised on the table size and its average
>>> tuple width might expose the most suspicious tables (in terms of
>>> table statistics).
>>> 2. DML Skew. Dividing rev_all_visible_pages by the number of tuple
>>> updates/deletes, normalised by the average table and tuple sizes,
>>> might indicate whether changes are localised within the table.
>>> 3. IndexOnlyScan effectiveness. Considering the speed of
>>> rev_all_visible_pages change, normalised to the value of the
>>> relallvisible statistic, we may detect tables where Index-Only Scan
>>> might be inefficiently used.
>>
>> With the parameter that was included before (pg_class_relallfrozen
>> and relallvisible
>> https://github.com/MasaoFujii/postgresql/commit/99f8f3fbbc8f743290844e8c676d39dad11c5d5d)
>> in the pg_stat_tables, I think I can provide isolation test to prove
>> it - I can use my isolation test
>> vacuum-extending-in-repetable-read.spec that I have added in the
>> extension (ext_vacuum_statistics). What do you think?
>
> I've prepared the test. Do you think it would make sense to include it
> in 0001?
>
I have added it in the 31th version for now and nothing else has been
changed (if you don't mind, exclude it).
From 486a29e6a22d43e2911eb849bdb3b3b39eefab91 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Fri, 13 Mar 2026 16:00:39 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 2 +
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 12 +-
src/include/pgstat.h | 17 +-
.../t/052_vacuum_extending_freeze_test.pl | 215 ++++++++++++++++++
src/test/regress/expected/rules.out | 6 +
9 files changed, 300 insertions(+), 2 deletions(-)
create mode 100644 src/test/recovery/t/052_vacuum_extending_freeze_test.pl
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index b77d189a500..fb656977b2e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4090,6 +4090,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-visible bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-visible, for example during an <command>INSERT</command>,
+ <command>UPDATE</command>, or <command>DELETE</command>.
+ A high rate of change in this counter means that index-only scans
+ on this table may frequently need to fall back to heap fetches,
+ and that vacuum must re-do visibility map work on those pages.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-frozen bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-frozen. A high value compared to the number of vacuum cycles
+ indicates that DML activity is frequently undoing the freezing work
+ performed by vacuum.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 3047bd46def..2e7c28ea307 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -161,6 +162,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 7553f31fef0..fa4c74bcd5d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -715,6 +715,8 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_dead_tuples(C.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(C.oid) as last_vacuum,
pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum,
pg_stat_get_last_analyze_time(C.oid) as last_analyze,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..bb26e97898d 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 73ca0bb0b7f..901f3dd55a1 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5e5e33f64fc..961337ce282 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12693,6 +12693,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index fff7ecc2533..04ccb3c06c2 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -156,6 +156,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -214,7 +216,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -447,6 +449,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -722,6 +726,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/recovery/t/052_vacuum_extending_freeze_test.pl b/src/test/recovery/t/052_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..384e123381f
--- /dev/null
+++ b/src/test/recovery/t/052_vacuum_extending_freeze_test.pl
@@ -0,0 +1,215 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats system using TAP
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+plan tests => 10;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('vacuum_extending_freeze_test');
+$node->init;
+
+# Configure the server for aggressive freezing behavior used by the test
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Enable necessary settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+# Polls statistics until the named columns exceed the provided
+# baseline values or until timeout.
+#
+# run_vacuum is a boolean (0 or 1) means we need to fetch frozen and visible pages
+# from pg_class table, otherwise we need to fetch frozen and visible pages from pg_stat_all_tables table.
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $run_vacuum = ($args{run_vacuum} or 0);
+ my $result_query;
+ my $sql;
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM vestat');
+
+ $sql = "
+ SELECT relallfrozen > 0
+ AND relallvisible > 0
+ FROM pg_class c
+ WHERE c.relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT rev_all_frozen_pages > 0
+ AND rev_all_visible_pages > 0
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ # sub-second sleep
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $relallvisible = 0;
+my $relallfrozen = 0;
+
+my $relallvisible_prev = 0;
+my $relallfrozen_prev = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT c.relallvisible, c.relallfrozen,
+ rev_all_visible_pages, rev_all_frozen_pages
+ FROM pg_class c
+ LEFT JOIN pg_stat_all_tables s ON s.relid = c.oid
+ WHERE c.relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($relallvisible, $relallfrozen, $rev_all_visible_pages, $rev_all_frozen_pages)
+ = split /\s+/, $base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+$node->safe_psql($dbname, q{
+ SELECT pg_stat_force_next_flush();
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 5000) AS g(x);
+ ANALYZE vestat;
+});
+
+# Poll the stats view until the expected deltas appear or timeout.
+$updated = wait_for_vacuum_stats(run_vacuum => 1);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (relallfrozen and relallvisible advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Verify initial statistics after vacuum
+#------------------------------------------------------------------------------
+ok($relallfrozen > $relallfrozen_prev, 'relallfrozen has increased');
+ok($relallvisible > $relallvisible_prev, 'relallvisible has increased');
+ok($rev_all_frozen_pages == 0, 'rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == 0, 'rev_all_visible_pages stay the same');
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+$relallfrozen_prev = $relallfrozen;
+$relallvisible_prev = $relallvisible;
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$node->safe_psql($dbname, 'SELECT pg_stat_force_next_flush()');
+
+# Poll until stats update or timeout.
+$updated = wait_for_vacuum_stats(run_vacuum => 0);
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Check updated statistics after backend activity
+#------------------------------------------------------------------------------
+
+ok($relallfrozen == $relallfrozen_prev, 'relallfrozen stay the same');
+ok($relallvisible == $relallvisible_prev, 'relallvisible stay the same');
+ok($rev_all_frozen_pages > 0, 'rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > 0, 'rev_all_visible_pages has increased');
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f4ee2bd7459..8dbf5ce34bb 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1834,6 +1834,8 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_dead_tuples(c.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum,
pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum,
pg_stat_get_last_analyze_time(c.oid) AS last_analyze,
@@ -2256,6 +2258,8 @@ pg_stat_sys_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
@@ -2311,6 +2315,8 @@ pg_stat_user_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v31-0001-Track-table-VM-stability.patch (19.2K, 2-v31-0001-Track-table-VM-stability.patch)
download | inline diff:
From 486a29e6a22d43e2911eb849bdb3b3b39eefab91 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Fri, 13 Mar 2026 16:00:39 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 2 +
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 12 +-
src/include/pgstat.h | 17 +-
.../t/052_vacuum_extending_freeze_test.pl | 215 ++++++++++++++++++
src/test/regress/expected/rules.out | 6 +
9 files changed, 300 insertions(+), 2 deletions(-)
create mode 100644 src/test/recovery/t/052_vacuum_extending_freeze_test.pl
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index b77d189a500..fb656977b2e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4090,6 +4090,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_visible_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-visible bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-visible, for example during an <command>INSERT</command>,
+ <command>UPDATE</command>, or <command>DELETE</command>.
+ A high rate of change in this counter means that index-only scans
+ on this table may frequently need to fall back to heap fetches,
+ and that vacuum must re-do visibility map work on those pages.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rev_all_frozen_pages</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for a
+ page of this table. The all-frozen bit is cleared by backend
+ processes when they modify a heap page that was previously marked
+ all-frozen. A high value compared to the number of vacuum cycles
+ indicates that DML activity is frequently undoing the freezing work
+ performed by vacuum.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 3047bd46def..2e7c28ea307 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -161,6 +162,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 7553f31fef0..fa4c74bcd5d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -715,6 +715,8 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_dead_tuples(C.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(C.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(C.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(C.oid) as last_vacuum,
pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum,
pg_stat_get_last_analyze_time(C.oid) as last_analyze,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..bb26e97898d 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 73ca0bb0b7f..901f3dd55a1 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -106,6 +106,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_rev_all_frozen_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_frozen_pages)
+
+/* pg_stat_get_rev_all_visible_pages */
+PG_STAT_GET_RELENTRY_INT64(rev_all_visible_pages)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5e5e33f64fc..961337ce282 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12693,6 +12693,16 @@
prosrc => 'hashoid8' },
{ oid => '8281', descr => 'hash',
proname => 'hashoid8extended', prorettype => 'int8',
- proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+ proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_visible_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_visible_pages' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map was removed for pages of table',
+ proname => 'pg_stat_get_rev_all_frozen_pages', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_rev_all_frozen_pages' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index fff7ecc2533..04ccb3c06c2 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -156,6 +156,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
} PgStat_TableCounts;
/* ----------
@@ -214,7 +216,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -447,6 +449,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -722,6 +726,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/recovery/t/052_vacuum_extending_freeze_test.pl b/src/test/recovery/t/052_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..384e123381f
--- /dev/null
+++ b/src/test/recovery/t/052_vacuum_extending_freeze_test.pl
@@ -0,0 +1,215 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats system using TAP
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+plan tests => 10;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('vacuum_extending_freeze_test');
+$node->init;
+
+# Configure the server for aggressive freezing behavior used by the test
+$node->append_conf('postgresql.conf', q{
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Enable necessary settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+# Polls statistics until the named columns exceed the provided
+# baseline values or until timeout.
+#
+# run_vacuum is a boolean (0 or 1) means we need to fetch frozen and visible pages
+# from pg_class table, otherwise we need to fetch frozen and visible pages from pg_stat_all_tables table.
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $run_vacuum = ($args{run_vacuum} or 0);
+ my $result_query;
+ my $sql;
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM vestat');
+
+ $sql = "
+ SELECT relallfrozen > 0
+ AND relallvisible > 0
+ FROM pg_class c
+ WHERE c.relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT rev_all_frozen_pages > 0
+ AND rev_all_visible_pages > 0
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ # sub-second sleep
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $relallvisible = 0;
+my $relallfrozen = 0;
+
+my $relallvisible_prev = 0;
+my $relallfrozen_prev = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT c.relallvisible, c.relallfrozen,
+ rev_all_visible_pages, rev_all_frozen_pages
+ FROM pg_class c
+ LEFT JOIN pg_stat_all_tables s ON s.relid = c.oid
+ WHERE c.relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($relallvisible, $relallfrozen, $rev_all_visible_pages, $rev_all_frozen_pages)
+ = split /\s+/, $base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+$node->safe_psql($dbname, q{
+ SELECT pg_stat_force_next_flush();
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 5000) AS g(x);
+ ANALYZE vestat;
+});
+
+# Poll the stats view until the expected deltas appear or timeout.
+$updated = wait_for_vacuum_stats(run_vacuum => 1);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (relallfrozen and relallvisible advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_tables to update after $timeout seconds during vacuum";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Verify initial statistics after vacuum
+#------------------------------------------------------------------------------
+ok($relallfrozen > $relallfrozen_prev, 'relallfrozen has increased');
+ok($relallvisible > $relallvisible_prev, 'relallvisible has increased');
+ok($rev_all_frozen_pages == 0, 'rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == 0, 'rev_all_visible_pages stay the same');
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+$relallfrozen_prev = $relallfrozen;
+$relallvisible_prev = $relallvisible;
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$node->safe_psql($dbname, 'SELECT pg_stat_force_next_flush()');
+
+# Poll until stats update or timeout.
+$updated = wait_for_vacuum_stats(run_vacuum => 0);
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for pg_stats_vacuum_* update after $timeout seconds";
+
+#------------------------------------------------------------------------------
+# Snapshot current statistics for later comparison
+#------------------------------------------------------------------------------
+
+fetch_vacuum_stats();
+
+#------------------------------------------------------------------------------
+# Check updated statistics after backend activity
+#------------------------------------------------------------------------------
+
+ok($relallfrozen == $relallfrozen_prev, 'relallfrozen stay the same');
+ok($relallvisible == $relallvisible_prev, 'relallvisible stay the same');
+ok($rev_all_frozen_pages > 0, 'rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > 0, 'rev_all_visible_pages has increased');
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f4ee2bd7459..8dbf5ce34bb 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1834,6 +1834,8 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_dead_tuples(c.oid) AS n_dead_tup,
pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze,
pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum,
+ pg_stat_get_rev_all_visible_pages(c.oid) AS rev_all_visible_pages,
+ pg_stat_get_rev_all_frozen_pages(c.oid) AS rev_all_frozen_pages,
pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum,
pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum,
pg_stat_get_last_analyze_time(c.oid) AS last_analyze,
@@ -2256,6 +2258,8 @@ pg_stat_sys_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
@@ -2311,6 +2315,8 @@ pg_stat_user_tables| SELECT relid,
n_dead_tup,
n_mod_since_analyze,
n_ins_since_vacuum,
+ rev_all_visible_pages,
+ rev_all_frozen_pages,
last_vacuum,
last_autovacuum,
last_analyze,
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-16 08:45 Andrei Lepikhov <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 1 reply; 46+ messages in thread
From: Andrei Lepikhov @ 2026-03-16 08:45 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; Alena Rybakina <[email protected]>; +Cc: pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 15/3/26 18:18, Andrey Borodin wrote:
>
>
>> On 13 Mar 2026, at 18:04, Alena Rybakina <[email protected]> wrote:
>
> I've decided to take a look into v31.
>
> Overall idea of tracking VM dynamics seems good to me.
>
> But the column naming for rev_all_visible_pages and rev_all_frozen_pages
> seems strange to me. I've skimmed the thread but could not figure out what
> "rev_" stands for. Revisions? Revolutions? Reviews?
I suppose 'revert' is the exact term here. Someone decided to set the
flag, and we reverted his decision. Does this make sense to you? Anyway,
I always leave it in the natives' (and committers') hands.
>
> Is there a reason why you break "SELECT * FROM pg_stat_all_tables" for
> an existing software? IMO even if we want these columns in this exact view
> - they ought to be appended to the end of the column list.
Please specify what you mean by this 'break'?
The relational model has never guaranteed a specific order of columns
returned unless you specify their names explicitly as a list. I think it
is good if someone found a flaw in their application, depending on the
wildcard order. So, I organised the elements according to their logical
order.
What's more? If you check the history of this VIEW, you will find that
it has always been updated in logical order. Please explain your point
if I misunderstood it.
>
> Some nits about the code.
I doubt if we need a test for these parameters - they reflect the
physical structure of the storage and might be unstable. But anyway, it
should be better to live in isolation tests, as similar statistics.
Thanks for your efforts!
--
regards, Andrei Lepikhov,
pgEdge
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-16 12:07 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
1 sibling, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2026-03-16 12:07 UTC (permalink / raw)
To: Андрей Зубков <[email protected]>; Andrey Borodin <[email protected]>; +Cc: Andrei Lepikhov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
Hi! Thank you for your attention to this patch!
On 16.03.2026 11:34, Андрей Зубков wrote:
> Really it was "revocations", but I'm agree with Andrey that naming
> isn't clear. *_vm_cleared looks better, but talking about naming here
> "vm" meaning is not clear. I think it will be understood as visibility
> map, but it is "mark" really. Maybe "*_pages_marks_cleared" will be
> better?
>
> Also a macro in pgstat.h:733 and pgstat.h:738 still holds "_rev_".
Good catch, fixed.
>
> I think the docs description needs a little correction:
>
> - visible_pages_vm_cleared. I think listing of possible DML operations
> is not needed here, also it seems a high rate of this counter has no
> direct relation to the index only scans because we can have very
> agressive vacuum on a table that will do the opposite. It will hold
> few pages without visibility marks constantly but with the cost
> of high visible_pages_vm_cleared rate. My proposition follows:
>
> Number of times the all-visible bit in the
> <link linkend="storage-vm">visibility map</link> was cleared for a
> pages of this table. The all-visible bit of a heap page is cleared
> every time backend process modifies a page previously marked
> all-visible by vacuum. Vacuum process must process page once again
> on the next run. A high rate of change of this counter means that
> vacuum should re-do its work on this table.
>
> - frozen_pages_vm_cleared:
>
> Number of times the all-frozen bit in the
> <link linkend="storage-vm">visibility map</link> was cleared for a
> pages of this table. The all-frozen bit of a heap page is cleared
> every time backend process modifies a page previously marked
> all-frozen by vacuum. Vacuum process must process page once again on
> the next freeze run on this table.
I agree, this description is clearer. Fixed.
-----------
Best regards,
Alena Rybakina
From bd7cbd4450512aaf640156e977faa28e5095d33a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 16 Mar 2026 14:55:45 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 ++++++++
src/backend/access/heap/visibilitymap.c | 10 +++
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 ++
src/include/catalog/pg_proc.dat | 10 +++
src/include/pgstat.h | 17 ++++-
.../expected/vacuum-extending-freeze.out | 50 +++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 73 +++++++++++++++++++
src/test/regress/expected/rules.out | 12 ++-
11 files changed, 212 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 9c5c6dc490f..0b27558686e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4258,6 +4258,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_pages_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible bit of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by <command>VACUUM</command>. The
+ page must then be processed again by <command>VACUUM</command> on a
+ subsequent run. A high rate of change in this counter means that
+ <command>VACUUM</command> has to repeatedly re-process pages of this
+ table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_pages_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen bit of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by <command>VACUUM</command>. The page must then
+ be processed again by <command>VACUUM</command> on the next freeze
+ run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..7b3ab6244d0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_pages_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_pages_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 90d48bc9c80..9ff013ac797 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -741,7 +741,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_pages_cleared(C.oid) AS visible_pages_cleared,
+ pg_stat_get_frozen_pages_cleared(C.oid) AS frozen_pages_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..78936aca82e 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_pages_cleared += lstats->counts.visible_pages_cleared;
+ tabentry->frozen_pages_cleared += lstats->counts.frozen_pages_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5ac022274a7..d50b7233c0e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_pages_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_pages_cleared)
+
+/* pg_stat_get_frozen_pages_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_pages_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..b52e463e63f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12833,4 +12833,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_pages_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_pages_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_pages_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_pages_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..8116d0959de 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_pages_cleared;
+ PgStat_Counter frozen_pages_cleared;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +219,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -450,6 +452,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_pages_cleared;
+ PgStat_Counter frozen_pages_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -725,6 +729,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_visible_pages_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_pages_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_pages_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_pages_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..58b51570e5e
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,50 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1_initial_vacuum s2_vacuum s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_initial_vacuum:
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum:
+ VACUUM vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT relallfrozen > 0 AS relallfrozen_pos,
+ relallvisible > 0 AS relallvisible_pos
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+relallfrozen_pos|relallvisible_pos
+----------------+-----------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT visible_pages_cleared > 0 AS visible_pages_cleared,
+ frozen_pages_cleared > 0 AS frozen_pages_cleared
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat';
+
+visible_pages_cleared|frozen_pages_cleared
+---------------------+--------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 4e466580cd4..81e68f85d88 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -124,3 +124,4 @@ test: serializable-parallel-2
test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..b8f8c177595
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,73 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+
+setup
+{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i FROM generate_series(1, 5000) AS g(i);
+
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_initial_vacuum
+{
+ SELECT pg_stat_force_next_flush();
+}
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT relallfrozen > 0 AS relallfrozen_pos,
+ relallvisible > 0 AS relallvisible_pos
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT visible_pages_cleared > 0 AS visible_pages_cleared,
+ frozen_pages_cleared > 0 AS frozen_pages_cleared
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat';
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_vacuum
+{
+ VACUUM vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s1_initial_vacuum
+ s2_vacuum
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 71d7262049e..b36b551d877 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_pages_cleared(c.oid) AS visible_pages_cleared,
+ pg_stat_get_frozen_pages_cleared(c.oid) AS frozen_pages_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2298,7 +2300,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_pages_cleared,
+ frozen_pages_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2353,7 +2357,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_pages_cleared,
+ frozen_pages_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v34-0001-Track-table-VM-stability.patch (15.5K, 2-v34-0001-Track-table-VM-stability.patch)
download | inline diff:
From bd7cbd4450512aaf640156e977faa28e5095d33a Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 16 Mar 2026 14:55:45 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 ++++++++
src/backend/access/heap/visibilitymap.c | 10 +++
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 ++
src/include/catalog/pg_proc.dat | 10 +++
src/include/pgstat.h | 17 ++++-
.../expected/vacuum-extending-freeze.out | 50 +++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 73 +++++++++++++++++++
src/test/regress/expected/rules.out | 12 ++-
11 files changed, 212 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 9c5c6dc490f..0b27558686e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4258,6 +4258,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_pages_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible bit of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by <command>VACUUM</command>. The
+ page must then be processed again by <command>VACUUM</command> on a
+ subsequent run. A high rate of change in this counter means that
+ <command>VACUUM</command> has to repeatedly re-process pages of this
+ table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_pages_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen bit in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen bit of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by <command>VACUUM</command>. The page must then
+ be processed again by <command>VACUUM</command> on the next freeze
+ run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..7b3ab6244d0 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -92,6 +92,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -163,6 +164,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_pages_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_pages_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 90d48bc9c80..9ff013ac797 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -741,7 +741,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_pages_cleared(C.oid) AS visible_pages_cleared,
+ pg_stat_get_frozen_pages_cleared(C.oid) AS frozen_pages_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..78936aca82e 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_pages_cleared += lstats->counts.visible_pages_cleared;
+ tabentry->frozen_pages_cleared += lstats->counts.frozen_pages_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5ac022274a7..d50b7233c0e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -107,6 +107,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_pages_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_pages_cleared)
+
+/* pg_stat_get_frozen_pages_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_pages_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 361e2cfffeb..b52e463e63f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12833,4 +12833,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible pages in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_pages_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_pages_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen pages in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_pages_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_pages_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 216b93492ba..8116d0959de 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -159,6 +159,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_pages_cleared;
+ PgStat_Counter frozen_pages_cleared;
} PgStat_TableCounts;
/* ----------
@@ -217,7 +219,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBB
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
typedef struct PgStat_ArchiverStats
{
@@ -450,6 +452,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_pages_cleared;
+ PgStat_Counter frozen_pages_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -725,6 +729,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen bits in visibility map */
+#define pgstat_count_visible_pages_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_pages_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_pages_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_pages_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..58b51570e5e
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,50 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1_initial_vacuum s2_vacuum s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_initial_vacuum:
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum:
+ VACUUM vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT relallfrozen > 0 AS relallfrozen_pos,
+ relallvisible > 0 AS relallvisible_pos
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+relallfrozen_pos|relallvisible_pos
+----------------+-----------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT visible_pages_cleared > 0 AS visible_pages_cleared,
+ frozen_pages_cleared > 0 AS frozen_pages_cleared
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat';
+
+visible_pages_cleared|frozen_pages_cleared
+---------------------+--------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 4e466580cd4..81e68f85d88 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -124,3 +124,4 @@ test: serializable-parallel-2
test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..b8f8c177595
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,73 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+
+setup
+{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i FROM generate_series(1, 5000) AS g(i);
+
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_initial_vacuum
+{
+ SELECT pg_stat_force_next_flush();
+}
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT relallfrozen > 0 AS relallfrozen_pos,
+ relallvisible > 0 AS relallvisible_pos
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT visible_pages_cleared > 0 AS visible_pages_cleared,
+ frozen_pages_cleared > 0 AS frozen_pages_cleared
+ FROM pg_stat_all_tables
+ WHERE relname = 'vestat';
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_vacuum
+{
+ VACUUM vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s1_initial_vacuum
+ s2_vacuum
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 71d7262049e..b36b551d877 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_pages_cleared(c.oid) AS visible_pages_cleared,
+ pg_stat_get_frozen_pages_cleared(c.oid) AS frozen_pages_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2298,7 +2300,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_pages_cleared,
+ frozen_pages_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2353,7 +2357,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_pages_cleared,
+ frozen_pages_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-16 12:11 Alena Rybakina <[email protected]>
parent: Andrei Lepikhov <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2026-03-16 12:11 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; Andrey Borodin <[email protected]>; +Cc: pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
On 16.03.2026 11:45, Andrei Lepikhov wrote:
> On 15/3/26 18:18, Andrey Borodin wrote:
>>> On 13 Mar 2026, at 18:04, Alena Rybakina <[email protected]>
>>> wrote:
>>
>> I've decided to take a look into v31.
>>
>> Overall idea of tracking VM dynamics seems good to me.
>>
>> But the column naming for rev_all_visible_pages and rev_all_frozen_pages
>> seems strange to me. I've skimmed the thread but could not figure out
>> what
>> "rev_" stands for. Revisions? Revolutions? Reviews?
>
> I suppose 'revert' is the exact term here. Someone decided to set the
> flag, and we reverted his decision. Does this make sense to you?
> Anyway, I always leave it in the natives' (and committers') hands.
I think renaming them to 'cleared' helps avoid the confusion.
I have adopted the names proposed by A. Zubkov in v34.
>> Some nits about the code.
>
> I doubt if we need a test for these parameters - they reflect the
> physical structure of the storage and might be unstable. But anyway,
> it should be better to live in isolation tests, as similar statistics.
>
I moved the tests there. Regression tests are unfortunately not an
option because the statistics are not stable.
If the isolation test turns out to be unstable again, I'll move them
back to the TAP tests as I initially implemented,
following A. Borodin's suggestion.
See the version in the
https://www.postgresql.org/message-id/767d28c9-2ae8-43df-9f2e-3e8785075115%40yandex.ru
--
-----------
Best regards,
Alena Rybakina
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-16 14:27 Andrey Borodin <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Andrey Borodin @ 2026-03-16 14:27 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Andrei Lepikhov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Andrei Zubkov <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
> On 16 Mar 2026, at 17:11, Alena Rybakina <[email protected]> wrote:
>
> I moved the tests there. Regression tests are unfortunately not an option because the statistics are not stable.
I think there's no need to test for correct numbers. It would be totally enough
to just for sane numbers or even any numbers at all.
If the test just invokes the function without segfaulting - that's already by far
better than no test at all.
Of course, test that verifies expected behavior is better, if it's possible.
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-17 15:27 Andrei Zubkov <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Andrei Zubkov @ 2026-03-17 15:27 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Andrey Borodin <[email protected]>; Andrei Lepikhov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
Alena Rybakina <[email protected]> writes:
Hi, Alena!
I have some thoughts about your descriptions in the docs (V34). It looks
clearer now, but in seems to me it still contains some inaccuracy..
1. Naming: The meaning of 'visible/frozen_pages_cleared' seems to me as
cleared pages rather then cleared marks... Maybe it should be
'visible_page_marks_cleared'?
2. Mention of a <command>VACUUM</command> in the docs may be understood
as related to manual VACUUM command only. However, autovacuum is
accounted as well.. I think we can use just term 'vacuum' here as a
facility rather than command.
--
regards, Andrei Zubkov
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-18 14:19 Andrei Zubkov <[email protected]>
parent: Andrei Zubkov <[email protected]>
0 siblings, 0 replies; 46+ messages in thread
From: Andrei Zubkov @ 2026-03-18 14:19 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: Andrey Borodin <[email protected]>; Andrei Lepikhov <[email protected]>; pgsql-hackers; Alexander Korotkov <[email protected]>; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
Alena Rybakina <[email protected]> writes:
> 1. Naming: The meaning of 'visible/frozen_pages_cleared' seems to me as
> cleared pages rather then cleared marks... Maybe it should be
> 'visible_page_marks_cleared'?
>
> I agree, this improves clarity. I've renamed it as proposed.
>
> 2. Mention of a <command>VACUUM</command> in the docs may be understood
> as related to manual VACUUM command only. However, autovacuum is
> accounted as well.. I think we can use just term 'vacuum' here as a
> facility rather than command.
>
> Fixed. Clarified that this applies to both manual
> <command>VACUUM</command> and autovacuum.
I think it is good now
>
> I also added an additional test scenario where one process holds a
> transaction open while another process deletes tuples. We expect that
> the all-visible and all-frozen flags, previously set by VACUUM, are
> cleared only after the deleting transaction commits and the changes
> become visible.
>
I'm happy with the patch now.
--
Best regards, Andrei Zubkov
Postgres Professional
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-03-30 06:13 Alena Rybakina <[email protected]>
parent: Andrey Borodin <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2026-03-30 06:13 UTC (permalink / raw)
To: Alexander Korotkov <[email protected]>; Andrey Borodin <[email protected]>; Andrei Lepikhov <[email protected]>; Andrei Zubkov <[email protected]>; +Cc: pgsql-hackers; Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Ilia Evdokimov <[email protected]>
Hi, all!
On 17.03.2026 21:11, Alena Rybakina wrote:
> I think last version is stable - it is in the isolation test. The last
> version is here
> https://www.postgresql.org/message-id/68939c47-fa0c-4198-853a-92d1390079da%40yandex.ru
>
Nothing special has been changed. I have rebased the patch because of
updated PGSTAT_FILE_FORMAT_ID.
-----------
Best regards,
Alena Rybakina
From 7cea0dfa3c30805797a0a3d6ca8f8ac9b617d4a8 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index bb75ed1069b..83c9e265624 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4362,6 +4362,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index e54018004db..855ea965583 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..78c0e6329bc 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 9185a8e6b83..2e2b6897d36 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0118e970dda..f6028006776 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12851,4 +12851,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 8e3549c3752..3a6d75892fa 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -753,6 +757,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 4e466580cd4..81e68f85d88 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -124,3 +124,4 @@ test: serializable-parallel-2
test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2b3cf6d8569..9036eb29988 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2304,7 +2306,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2359,7 +2363,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v36-0001-Track-table-VM-stability.patch (21.7K, 2-v36-0001-Track-table-VM-stability.patch)
download | inline diff:
From 7cea0dfa3c30805797a0a3d6ca8f8ac9b617d4a8 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index bb75ed1069b..83c9e265624 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4362,6 +4362,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index e54018004db..855ea965583 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index bc8c43b96aa..78c0e6329bc 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -879,6 +879,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 9185a8e6b83..2e2b6897d36 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0118e970dda..f6028006776 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12851,4 +12851,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 8e3549c3752..3a6d75892fa 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -753,6 +757,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 4e466580cd4..81e68f85d88 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -124,3 +124,4 @@ test: serializable-parallel-2
test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2b3cf6d8569..9036eb29988 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2304,7 +2306,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2359,7 +2363,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-04-14 11:10 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2026-04-14 11:10 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrey Borodin <[email protected]>; Andrei Zubkov <[email protected]>; Andrei Lepikhov <[email protected]>
On 30.03.2026 09:13, Alena Rybakina wrote:
> Hi, all!
>
> On 17.03.2026 21:11, Alena Rybakina wrote:
>
>> I think last version is stable - it is in the isolation test. The
>> last version is here
>> https://www.postgresql.org/message-id/68939c47-fa0c-4198-853a-92d1390079da%40yandex.ru
>>
> Nothing special has been changed. I have rebased the patch because of
> updated PGSTAT_FILE_FORMAT_ID.
>
I have rebased the patch.
--
-----------
Best regards,
Alena Rybakina
From 5d8eddaf00d2729c3bc20dfb20debff1833b240e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v37-0001-Track-table-VM-stability.patch (21.7K, 2-v37-0001-Track-table-VM-stability.patch)
download | inline diff:
From 5d8eddaf00d2729c3bc20dfb20debff1833b240e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-04-28 02:16 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 1 reply; 46+ messages in thread
From: Alena Rybakina @ 2026-04-28 02:16 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrey Borodin <[email protected]>; Andrei Zubkov <[email protected]>; Andrei Lepikhov <[email protected]>
Hi, all!
I have updated the core patch that implements the machinery for
collecting extended vacuum statistics (I didn't touch the first patch
that is ready for commit, only patches that are related to extension),
and rebased the ext_vacuum_statistics extension on top of it. The split
is intentional: the core only gathers metrics and hands them out, while
the actual storage and SQL-level access to the statistics live entirely
in the extension. If the extension is not loaded, the overhead is
essentially zero - we only fill a small struct on the stack and do a
NULL check on the hook.
What was updated in the core
The core gains the machinery and the hook through which the extension
receives metrics after each vacuum.
The hook. A new hook has been added in pgstat - set_report_vacuum_hook.
It is fired once per vacuumed table and once per vacuumed index, plus
when forming the per-database aggregate. The extension registers its
handler in _PG_init and by default the hook is NULL, so without an
extension the core behaves exactly as before.
The set of statistics is the same as before. Common to tables, indexes
and the database - hits and misses in shared buffers, number of dirtied
and written pages, WAL volume, buffer read and write times, sleep time
spent in delay points, total wall-clock vacuum time (including I/O and
lock waits), counter of emergency anti-wraparound vacuums, number of
interrupts and removed tuples. Tables additionally report frozen tuples,
pages marked all-frozen / all-visible in the visibility map, number of
scanned and removed pages, number of index passes, etc. Indexes report
freed pages.
The least obvious part of the implementation is subtracting index
statistics from the table statistics. This is the bit worth
highlighting. The thing is that indexes are vacuumed before the heap,
and the buffer and WAL statistics that we capture at the heap level by
the end of the heap vacuum already include everything that was spent on
the indexes. If we simply expose the diff of pgBufferUsage/pgWalUsage
between start and end, the table ends up with double-counted pages/WAL:
once in its own report, and a second time inside the reports of its
indexes. This is especially noticeable with parallel index vacuum:
workers accumulate their usage in the leader only after they finish, so
without subtraction the heap report would receive the combined cost of
all workers as a "bonus".
To handle this, as each index finishes vacuuming, its counters are
accumulated into the state of the current operation, and at the moment
the heap report is built these sums are subtracted out. As a result, the
extension receives clean numbers: "this is what was actually spent on
the table itself", and separately "this is what was actually spent on
each index". The behaviour is idempotent for both serial and parallel
vacuum.
The ext_vacuum_statistics extension
The extension registers the hook handler and stores the received data
through the pgstat custom statistics infrastructure. That is, vacuum
counters are kept not in the extension's own files, but together with
the regular cumulative statistics - they survive a restart and are reset
together with pg_stat_reset_*. Access is provided through three views:
one for tables, one for indexes, and one with the per-database aggregate.
Filtering
This is where the main flexibility lives - the extension does not force
"collect everything", but lets you choose both what to track and which
metrics to keep.
By object type. You can limit collection to databases only (without
per-table detail), to tables only, or collect both. Among tables, you
can additionally filter system / user / all.
By an explicit list. An alternative to "by type" is a whitelist: you
turn the corresponding mode on, and the extension starts collecting
statistics only for the databases and tables that were explicitly
registered via add_track_database / add_track_relation (with matching
remove_* for removal). When the lists are off, the type filter is in
effect; when they are on, only the list applies. This is convenient when
you are interested in monitoring specific "hot" tables and do not want
to spend memory on statistics for everything else.
This list is persisted to disk, and there is one more non-trivial part
here. List changes are concurrent - multiple sessions may call
add_track_* simultaneously, plus there is an object-access hook that
cleans the entry on DROP. To avoid ending up with a torn file, access to
the list is serialized via a dedicated LWLock tranche (requested from a
shmem_request_hook), and the file itself is written atomically: first
into a temporary file, then fflush + pg_fsync + durable_rename. All I/O
return codes are checked; on error the temporary file is removed and the
real one is left untouched; PG_TRY/PG_CATCH guarantees cleanup on
ereport(ERROR). Reading the list takes the same lock in shared mode, so
a concurrent write cannot tear the load.
By metric category. There is also a GUC that takes a list and turns on
the categories of interest - buffers, WAL, general counters, timings (or
all). Unwanted categories are simply skipped on the hook handler side
and never make it into the pgstat entry, which reduces the overhead of
the handler itself. This is useful when, for example, only timings are
needed - in that case the extension does not waste time copying the
buffer and WAL fields.
Privileges. The add_track_* / remove_track_* functions require superuser
or pg_read_all_stats. At the SQL level, EXECUTE is revoked from PUBLIC
and granted only to pg_read_all_stats, so a regular user has no access
to mutating the list. The views are unrestricted, like regular statistics.
What is in the patches
0002-Machinery-for-grabbing-extended-vacuum-statistics.patch - the
machinery in the core plus the hook.
0003-ext_vacuum_statistics-...patch - the extension itself, filtering,
views, tests.
-----------
Best regards,
Alena Rybakina
Yandex Cloud
From 19f5a39f7e97d3fc2d18415ba2c51ffcd3b32f49 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH 1/3] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
From 3a5e0bd82578d1fea63d6bda229dc4d0b224684e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add infrastructure inside lazy vacuum to gather extended per-vacuum
metrics and expose them to extensions via a new hook. Core itself
does not persist these metrics — that is the job of an extension
(see ext_vacuum_statistics).
Statistics are gathered separately for tables and indexes according
to vacuum phases. The ExtVacReport union and type field distinguish
PGSTAT_EXTVAC_TABLE vs PGSTAT_EXTVAC_INDEX. Heap vacuum stats are
sent to the cumulative statistics system after vacuum has processed
the indexes. Database vacuum statistics aggregate per-table and
per-index statistics within the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
set_report_vacuum_hook (set_report_vacuum_hook_type) -- called
once per vacuumed relation/index with a PgStat_VacuumRelationCounts
payload tagged by ExtVacReportType (PGSTAT_EXTVAC_TABLE / _INDEX /
_DB / _INVALID).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 234 ++++++++++++++++++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 12 +
src/backend/utils/activity/pgstat_relation.c | 24 ++
src/include/commands/vacuum.h | 29 +++
src/include/pgstat.h | 69 ++++++
6 files changed, 367 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 39395aed0d5..e4d4c93d641 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -283,6 +283,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -410,6 +412,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -485,6 +496,166 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -643,7 +814,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_vacuum_min_duration >= 0));
@@ -660,6 +834,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -687,7 +864,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -803,6 +982,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -985,11 +1167,26 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+
pgstat_progress_end_command();
if (instrument)
@@ -2903,6 +3100,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3015,7 +3213,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3033,6 +3235,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3041,6 +3244,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3065,7 +3276,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3084,12 +3299,21 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 99d0db82ed7..a7fb73173f5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2561,6 +2564,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 41cefcfde54..200f12a2d1b 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1076,6 +1076,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -1084,6 +1086,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -1112,6 +1116,13 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1276,6 +1287,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 92e1f60a080..226d7aa06d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -272,6 +272,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 956d9cea36d..a925f7da992 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -21,9 +21,11 @@
#include "catalog/pg_class.h"
#include "catalog/pg_statistic.h"
#include "catalog/pg_type.h"
+#include "executor/instrument.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -354,6 +356,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7db36cf8add..8d934973dc1 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,64 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -703,6 +761,17 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
From cf8285d7557582d6995d58ca62599e7e47b20b1b Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 28 Apr 2026 03:43:29 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics: extension for extended vacuum
statistics
Introduce a new extension that collects extended per-vacuum
metrics via set_report_vacuum_hook and stores them through pgstat's
custom statistics infrastructure.
Tracking scope is controlled by GUCs:
* vacuum_statistics.enabled -- master switch
* vacuum_statistics.object_types -- databases / relations / all
* vacuum_statistics.track_relations -- system / user / all
* vacuum_statistics.track_{databases,relations}_from_list
-- restrict tracking to objects registered via
add_track_database() / add_track_relation();
removal via remove_track_*() and OAT_DROP hook
* vacuum_statistics.collect -- buffers / wal /
general / timing / all, consulted by ACCUM_IF() to skip
unwanted categories at run time
add_track_* / remove_track_* require superuser or pg_read_all_stats.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 ++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 272 ++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 ++++
.../t/054_vacuum_extending_gucs_test.pl | 279 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1387 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 ++++++
doc/src/sgml/filelist.sgml | 1 +
18 files changed, 3909 insertions(+)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7d91fe77db3..3140f2bf844 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..aa3a9ec9699
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,272 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Track-list mutation requires superuser or pg_read_all_stats; hide the
+-- functions from PUBLIC so the error is also produced for ordinary users
+-- before the C-level privilege check runs.
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) TO pg_read_all_stats;
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT STABLE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..4f8f025c63e
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_frozen_page_marks_cleared(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_visible_page_marks_cleared(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_frozen_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_visible_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..a195249842b
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,279 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra),
+ # Make pending stats visible to subsequent sessions without sleeping.
+ "SELECT pg_stat_force_next_flush();");
+ $node->safe_psql($db, $sql);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session. Assert not only that the
+ # row count is zero, but that the specific counters remain zero: a stray
+ # row with zero counters would otherwise pass a bare COUNT(*)=0 check.
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no rows when disabled');
+
+ my $sums = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_dirtied), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($sums, '0', 'no counters accumulated when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 6: vacuum_statistics.collect - per-category gating
+#
+# With collect='wal' only wal_* counters must advance; buffer, timing, and
+# general categories must stay at zero. With collect='buffers' the inverse
+# holds. Unknown tokens must be rejected by the check-hook.
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.collect' => sub {
+ # wal-only: WAL counters should accumulate, buffers/timing/general should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'wal'"],
+ modify => 1,
+ });
+
+ my $wal = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal, 't', "collect='wal': wal_records accumulated");
+
+ my $other = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0)
+ + COALESCE(SUM(total_time), 0)
+ + COALESCE(SUM(tuples_deleted), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($other, '0',
+ "collect='wal': buffer/timing/general counters not accumulated");
+
+ # buffers-only: buffer counters should advance, WAL should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'buffers'"],
+ modify => 1,
+ });
+
+ my $buf = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($buf, 't', "collect='buffers': buffer counters accumulated");
+
+ my $wal_off = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal_off, '0',
+ "collect='buffers': WAL counters not accumulated");
+
+ # Unknown category must be rejected by the check-hook.
+ my ($ret, $stdout, $stderr) = $node->psql($dbname,
+ "SET vacuum_statistics.collect = 'nope'");
+ isnt($ret, 0, "collect='nope': rejected by check-hook");
+ like($stderr, qr/Unrecognized category "nope"/,
+ "collect='nope': errdetail names the offending token");
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..75d1bd2cf06
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1387 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_authid.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+#include "utils/tuplestore.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/*
+ * Bit flags for evs_collect_mask. Each category groups counters that can be
+ * accumulated (or skipped) together, letting users reduce overhead at run
+ * time by turning off categories they don't need.
+ */
+#define EVS_COLLECT_BUFFERS 0x1 /* blks_*, blk_*_time */
+#define EVS_COLLECT_WAL 0x2 /* wal_records, wal_fpi, wal_bytes */
+#define EVS_COLLECT_GENERAL 0x4 /* tuples_deleted, pages_*, vm_*,
+ * wraparound_failsafe_count,
+ * interrupts_count */
+#define EVS_COLLECT_TIMING 0x8 /* delay_time, total_time */
+#define EVS_COLLECT_ALL (EVS_COLLECT_BUFFERS | EVS_COLLECT_WAL | \
+ EVS_COLLECT_GENERAL | EVS_COLLECT_TIMING)
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+static char *evs_collect = "all"; /* categories to collect */
+static int evs_collect_mask = EVS_COLLECT_ALL;
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+static shmem_request_hook_type prev_shmem_request_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+static void evs_shmem_request(void);
+
+/* Hash tables for track_databases and track_relations_list (backend-local) */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+/*
+ * Named LWLock tranche protecting the on-disk track file and serializing
+ * backend-local reloads/saves across concurrent backends.
+ */
+#define EVS_TRACK_TRANCHE_NAME "ext_vacuum_statistics_track"
+static LWLock *evs_track_lock = NULL;
+
+static inline LWLock *
+evs_get_track_lock(void)
+{
+ if (evs_track_lock == NULL)
+ evs_track_lock = &GetNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME)->lock;
+ return evs_track_lock;
+}
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/*
+ * Accumulate a single counter only if its category is enabled in
+ * evs_collect_mask. Parentheses around every argument: the macro is invoked
+ * from expression contexts and with expressions as the destination pointer.
+ */
+#define ACCUM_IF(dst, src, field, cat) \
+ do { \
+ if ((evs_collect_mask) & (cat)) \
+ ((dst))->field += ((src))->field; \
+ } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_dirtied, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_written, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_fetched, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_read_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_write_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, delay_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, total_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, wal_records, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_fpi, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_bytes, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wraparound_failsafe_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, interrupts_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, tuples_deleted, EVS_COLLECT_GENERAL);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/*
+ * GUC check hooks: validate the string and compute the bitmask into *extra.
+ * Rejecting unknown values here prevents silent fall-through to "all".
+ */
+static bool
+evs_track_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "databases") == 0)
+ *bits = EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "relations") == 0)
+ *bits = EVS_TRACK_RELATIONS;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"databases\", \"relations\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ evs_track_bits = *((int *) extra);
+}
+
+static bool
+evs_track_relations_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+ else if (strcmp(*newval, "system") == 0)
+ *bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(*newval, "user") == 0)
+ *bits = EVS_FILTER_USER;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"system\", \"user\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ evs_track_relations_bits = *((int *) extra);
+}
+
+/*
+ * Check hook for vacuum_statistics.collect.
+ *
+ * Accepts a comma- or whitespace-separated list of category names
+ * (buffers, wal, general, timing) or the shorthand "all". Computes the
+ * matching bitmask once and stashes it in *extra; the assign hook just
+ * copies it into evs_collect_mask. Unknown tokens are rejected so the
+ * setting cannot silently collapse to the "all" default.
+ */
+static bool
+evs_collect_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *mask;
+ char *copy;
+ char *p;
+ char *tok;
+ int accum = 0;
+ bool saw_all = false;
+
+ if (*newval == NULL)
+ return false;
+
+ mask = (int *) guc_malloc(LOG, sizeof(int));
+ if (!mask)
+ return false;
+
+ /* Empty string means "all", matching the default behavior. */
+ if ((*newval)[0] == '\0')
+ {
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+ }
+
+ copy = pstrdup(*newval);
+ for (p = copy; (tok = strtok(p, " \t,")) != NULL; p = NULL)
+ {
+ if (pg_strcasecmp(tok, "all") == 0)
+ saw_all = true;
+ else if (pg_strcasecmp(tok, "buffers") == 0)
+ accum |= EVS_COLLECT_BUFFERS;
+ else if (pg_strcasecmp(tok, "wal") == 0)
+ accum |= EVS_COLLECT_WAL;
+ else if (pg_strcasecmp(tok, "general") == 0)
+ accum |= EVS_COLLECT_GENERAL;
+ else if (pg_strcasecmp(tok, "timing") == 0)
+ accum |= EVS_COLLECT_TIMING;
+ else
+ {
+ /*
+ * GUC_check_errdetail formats the message immediately, but tok
+ * points into copy; emit the detail first, then free the
+ * scratch buffer so the formatted string is already stashed in
+ * GUC_check_errdetail_string.
+ */
+ GUC_check_errdetail("Unrecognized category \"%s\" in vacuum_statistics.collect; "
+ "allowed values are \"all\", \"buffers\", \"wal\", \"general\", \"timing\".",
+ tok);
+ pfree(copy);
+ guc_free(mask);
+ return false;
+ }
+ }
+ pfree(copy);
+
+ *mask = saw_all ? EVS_COLLECT_ALL : accum;
+ if (*mask == 0)
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+}
+
+static void
+evs_collect_assign_hook(const char *newval, void *extra)
+{
+ evs_collect_mask = *((int *) extra);
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0,
+ evs_track_check_hook,
+ evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0,
+ evs_track_relations_check_hook,
+ evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.collect",
+ "Statistics categories to collect.",
+ "Comma- or whitespace-separated list of: "
+ "\"buffers\", \"wal\", \"general\", \"timing\"; "
+ "or \"all\" for every category (default).",
+ &evs_collect, "all",
+ PGC_SUSET, 0,
+ evs_collect_check_hook,
+ evs_collect_assign_hook, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = evs_shmem_request;
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+static void
+evs_shmem_request(void)
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
+ RequestNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME, 1);
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ LWLock *lock = evs_get_track_lock();
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ LWLock *lock = evs_get_track_lock();
+ bool found;
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+/*
+ * Initialize the backend-local tracking hashes and load their contents
+ * from the on-disk file.
+ *
+ * The hashes are per-backend, so no lock is needed to protect them from
+ * other processes; however, another backend may be concurrently rewriting
+ * the track file, so we take a shared lock for the file read.
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+ LWLock *lock;
+ bool need_load;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ lock = evs_get_track_lock();
+
+ if (evs_track_databases_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_databases_hash =
+ hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ if (evs_track_relations_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_relations_hash =
+ hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ need_load = !LWLockHeldByMe(lock);
+ if (need_load)
+ LWLockAcquire(lock, LW_SHARED);
+ PG_TRY();
+ {
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+ }
+ PG_FINALLY();
+ {
+ if (need_load)
+ LWLockRelease(lock);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Load track lists from disk into the backend-local hashes.
+ *
+ * Caller must hold evs_track_lock at least in shared mode, since the file
+ * may be concurrently rewritten by another backend.
+ */
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[MAXPGPATH];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open track file \"%s\": %m", path)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ size_t len = strlen(buf);
+
+ /* Reject unterminated lines (longer than buffer) as corruption. */
+ if (len > 0 && buf[len - 1] != '\n' && !feof(fp))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("line too long in track file \"%s\"", path)));
+
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+
+ if (ferror(fp))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read track file \"%s\": %m", path)));
+ }
+ PG_FINALLY();
+ {
+ FreeFile(fp);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Atomically rewrite the track file. Caller must hold evs_track_lock
+ * in exclusive mode.
+ */
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+ bool failed = false;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s.tmp", path);
+
+ fp = AllocateFile(tmppath, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create track file \"%s\": %m", tmppath)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ if (fputs("[databases]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ if (fprintf(fp, "%u\n", *entry) < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fputs("[relations]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ int rc;
+
+ if (OidIsValid(rel_entry->dboid))
+ rc = fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ rc = fprintf(fp, "0 %u\n", rel_entry->reloid);
+ if (rc < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fflush(fp) != 0)
+ failed = true;
+
+ if (!failed)
+ {
+ int fd = fileno(fp);
+
+ if (fd >= 0 && pg_fsync(fd) != 0)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not fsync track file \"%s\": %m",
+ tmppath)));
+ }
+ }
+ PG_CATCH();
+ {
+ FreeFile(fp);
+ (void) unlink(tmppath);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (FreeFile(fp) != 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close track file \"%s\": %m", tmppath)));
+ failed = true;
+ }
+
+ if (failed)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write track file \"%s\": %m", tmppath)));
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ return;
+ }
+
+ if (durable_rename(tmppath, path, LOG) != 0)
+ {
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ }
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+/*
+ * Mutating track-list entry points: require server-wide privilege, since
+ * the underlying lists steer tracking for every backend.
+ */
+static void
+evs_require_track_privilege(const char *funcname)
+{
+ if (!superuser() && !has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS))
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for function %s", funcname),
+ errhint("Only superusers and members of pg_read_all_stats "
+ "may change the vacuum statistics track list.")));
+}
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid,
+ InvalidOid, NULL);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index ebb7f83d8c5..d7dc0fd07f0 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index b9b03654aad..2a38f9042bb 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 25a85082759..85d721467c0 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v38-0001-Track-table-VM-stability.patch (21.7K, 3-v38-0001-Track-table-VM-stability.patch)
download | inline diff:
From 19f5a39f7e97d3fc2d18415ba2c51ffcd3b32f49 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH 1/3] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
[text/plain] v38-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch (25.0K, 4-v38-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch)
download | inline diff:
From 3a5e0bd82578d1fea63d6bda229dc4d0b224684e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add infrastructure inside lazy vacuum to gather extended per-vacuum
metrics and expose them to extensions via a new hook. Core itself
does not persist these metrics — that is the job of an extension
(see ext_vacuum_statistics).
Statistics are gathered separately for tables and indexes according
to vacuum phases. The ExtVacReport union and type field distinguish
PGSTAT_EXTVAC_TABLE vs PGSTAT_EXTVAC_INDEX. Heap vacuum stats are
sent to the cumulative statistics system after vacuum has processed
the indexes. Database vacuum statistics aggregate per-table and
per-index statistics within the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
set_report_vacuum_hook (set_report_vacuum_hook_type) -- called
once per vacuumed relation/index with a PgStat_VacuumRelationCounts
payload tagged by ExtVacReportType (PGSTAT_EXTVAC_TABLE / _INDEX /
_DB / _INVALID).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 234 ++++++++++++++++++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 12 +
src/backend/utils/activity/pgstat_relation.c | 24 ++
src/include/commands/vacuum.h | 29 +++
src/include/pgstat.h | 69 ++++++
6 files changed, 367 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 39395aed0d5..e4d4c93d641 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -283,6 +283,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -410,6 +412,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -485,6 +496,166 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -643,7 +814,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_vacuum_min_duration >= 0));
@@ -660,6 +834,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -687,7 +864,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -803,6 +982,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -985,11 +1167,26 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+
pgstat_progress_end_command();
if (instrument)
@@ -2903,6 +3100,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3015,7 +3213,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3033,6 +3235,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3041,6 +3244,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3065,7 +3276,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3084,12 +3299,21 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 99d0db82ed7..a7fb73173f5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2561,6 +2564,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 41cefcfde54..200f12a2d1b 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1076,6 +1076,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -1084,6 +1086,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -1112,6 +1116,13 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1276,6 +1287,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 92e1f60a080..226d7aa06d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -272,6 +272,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 956d9cea36d..a925f7da992 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -21,9 +21,11 @@
#include "catalog/pg_class.h"
#include "catalog/pg_statistic.h"
#include "catalog/pg_type.h"
+#include "executor/instrument.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -354,6 +356,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7db36cf8add..8d934973dc1 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,64 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -703,6 +761,17 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
[text/plain] v38-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch (145.2K, 5-v38-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch)
download | inline diff:
From cf8285d7557582d6995d58ca62599e7e47b20b1b Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 28 Apr 2026 03:43:29 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics: extension for extended vacuum
statistics
Introduce a new extension that collects extended per-vacuum
metrics via set_report_vacuum_hook and stores them through pgstat's
custom statistics infrastructure.
Tracking scope is controlled by GUCs:
* vacuum_statistics.enabled -- master switch
* vacuum_statistics.object_types -- databases / relations / all
* vacuum_statistics.track_relations -- system / user / all
* vacuum_statistics.track_{databases,relations}_from_list
-- restrict tracking to objects registered via
add_track_database() / add_track_relation();
removal via remove_track_*() and OAT_DROP hook
* vacuum_statistics.collect -- buffers / wal /
general / timing / all, consulted by ACCUM_IF() to skip
unwanted categories at run time
add_track_* / remove_track_* require superuser or pg_read_all_stats.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 ++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 272 ++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 ++++
.../t/054_vacuum_extending_gucs_test.pl | 279 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1387 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 ++++++
doc/src/sgml/filelist.sgml | 1 +
18 files changed, 3909 insertions(+)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7d91fe77db3..3140f2bf844 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..aa3a9ec9699
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,272 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Track-list mutation requires superuser or pg_read_all_stats; hide the
+-- functions from PUBLIC so the error is also produced for ordinary users
+-- before the C-level privilege check runs.
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) TO pg_read_all_stats;
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT STABLE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..4f8f025c63e
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_frozen_page_marks_cleared(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_visible_page_marks_cleared(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_frozen_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_visible_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..a195249842b
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,279 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra),
+ # Make pending stats visible to subsequent sessions without sleeping.
+ "SELECT pg_stat_force_next_flush();");
+ $node->safe_psql($db, $sql);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session. Assert not only that the
+ # row count is zero, but that the specific counters remain zero: a stray
+ # row with zero counters would otherwise pass a bare COUNT(*)=0 check.
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no rows when disabled');
+
+ my $sums = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_dirtied), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($sums, '0', 'no counters accumulated when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 6: vacuum_statistics.collect - per-category gating
+#
+# With collect='wal' only wal_* counters must advance; buffer, timing, and
+# general categories must stay at zero. With collect='buffers' the inverse
+# holds. Unknown tokens must be rejected by the check-hook.
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.collect' => sub {
+ # wal-only: WAL counters should accumulate, buffers/timing/general should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'wal'"],
+ modify => 1,
+ });
+
+ my $wal = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal, 't', "collect='wal': wal_records accumulated");
+
+ my $other = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0)
+ + COALESCE(SUM(total_time), 0)
+ + COALESCE(SUM(tuples_deleted), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($other, '0',
+ "collect='wal': buffer/timing/general counters not accumulated");
+
+ # buffers-only: buffer counters should advance, WAL should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'buffers'"],
+ modify => 1,
+ });
+
+ my $buf = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($buf, 't', "collect='buffers': buffer counters accumulated");
+
+ my $wal_off = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal_off, '0',
+ "collect='buffers': WAL counters not accumulated");
+
+ # Unknown category must be rejected by the check-hook.
+ my ($ret, $stdout, $stderr) = $node->psql($dbname,
+ "SET vacuum_statistics.collect = 'nope'");
+ isnt($ret, 0, "collect='nope': rejected by check-hook");
+ like($stderr, qr/Unrecognized category "nope"/,
+ "collect='nope': errdetail names the offending token");
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..75d1bd2cf06
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1387 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_authid.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+#include "utils/tuplestore.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/*
+ * Bit flags for evs_collect_mask. Each category groups counters that can be
+ * accumulated (or skipped) together, letting users reduce overhead at run
+ * time by turning off categories they don't need.
+ */
+#define EVS_COLLECT_BUFFERS 0x1 /* blks_*, blk_*_time */
+#define EVS_COLLECT_WAL 0x2 /* wal_records, wal_fpi, wal_bytes */
+#define EVS_COLLECT_GENERAL 0x4 /* tuples_deleted, pages_*, vm_*,
+ * wraparound_failsafe_count,
+ * interrupts_count */
+#define EVS_COLLECT_TIMING 0x8 /* delay_time, total_time */
+#define EVS_COLLECT_ALL (EVS_COLLECT_BUFFERS | EVS_COLLECT_WAL | \
+ EVS_COLLECT_GENERAL | EVS_COLLECT_TIMING)
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+static char *evs_collect = "all"; /* categories to collect */
+static int evs_collect_mask = EVS_COLLECT_ALL;
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+static shmem_request_hook_type prev_shmem_request_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+static void evs_shmem_request(void);
+
+/* Hash tables for track_databases and track_relations_list (backend-local) */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+/*
+ * Named LWLock tranche protecting the on-disk track file and serializing
+ * backend-local reloads/saves across concurrent backends.
+ */
+#define EVS_TRACK_TRANCHE_NAME "ext_vacuum_statistics_track"
+static LWLock *evs_track_lock = NULL;
+
+static inline LWLock *
+evs_get_track_lock(void)
+{
+ if (evs_track_lock == NULL)
+ evs_track_lock = &GetNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME)->lock;
+ return evs_track_lock;
+}
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/*
+ * Accumulate a single counter only if its category is enabled in
+ * evs_collect_mask. Parentheses around every argument: the macro is invoked
+ * from expression contexts and with expressions as the destination pointer.
+ */
+#define ACCUM_IF(dst, src, field, cat) \
+ do { \
+ if ((evs_collect_mask) & (cat)) \
+ ((dst))->field += ((src))->field; \
+ } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_dirtied, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_written, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_fetched, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_read_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_write_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, delay_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, total_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, wal_records, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_fpi, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_bytes, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wraparound_failsafe_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, interrupts_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, tuples_deleted, EVS_COLLECT_GENERAL);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/*
+ * GUC check hooks: validate the string and compute the bitmask into *extra.
+ * Rejecting unknown values here prevents silent fall-through to "all".
+ */
+static bool
+evs_track_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "databases") == 0)
+ *bits = EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "relations") == 0)
+ *bits = EVS_TRACK_RELATIONS;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"databases\", \"relations\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ evs_track_bits = *((int *) extra);
+}
+
+static bool
+evs_track_relations_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+ else if (strcmp(*newval, "system") == 0)
+ *bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(*newval, "user") == 0)
+ *bits = EVS_FILTER_USER;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"system\", \"user\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ evs_track_relations_bits = *((int *) extra);
+}
+
+/*
+ * Check hook for vacuum_statistics.collect.
+ *
+ * Accepts a comma- or whitespace-separated list of category names
+ * (buffers, wal, general, timing) or the shorthand "all". Computes the
+ * matching bitmask once and stashes it in *extra; the assign hook just
+ * copies it into evs_collect_mask. Unknown tokens are rejected so the
+ * setting cannot silently collapse to the "all" default.
+ */
+static bool
+evs_collect_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *mask;
+ char *copy;
+ char *p;
+ char *tok;
+ int accum = 0;
+ bool saw_all = false;
+
+ if (*newval == NULL)
+ return false;
+
+ mask = (int *) guc_malloc(LOG, sizeof(int));
+ if (!mask)
+ return false;
+
+ /* Empty string means "all", matching the default behavior. */
+ if ((*newval)[0] == '\0')
+ {
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+ }
+
+ copy = pstrdup(*newval);
+ for (p = copy; (tok = strtok(p, " \t,")) != NULL; p = NULL)
+ {
+ if (pg_strcasecmp(tok, "all") == 0)
+ saw_all = true;
+ else if (pg_strcasecmp(tok, "buffers") == 0)
+ accum |= EVS_COLLECT_BUFFERS;
+ else if (pg_strcasecmp(tok, "wal") == 0)
+ accum |= EVS_COLLECT_WAL;
+ else if (pg_strcasecmp(tok, "general") == 0)
+ accum |= EVS_COLLECT_GENERAL;
+ else if (pg_strcasecmp(tok, "timing") == 0)
+ accum |= EVS_COLLECT_TIMING;
+ else
+ {
+ /*
+ * GUC_check_errdetail formats the message immediately, but tok
+ * points into copy; emit the detail first, then free the
+ * scratch buffer so the formatted string is already stashed in
+ * GUC_check_errdetail_string.
+ */
+ GUC_check_errdetail("Unrecognized category \"%s\" in vacuum_statistics.collect; "
+ "allowed values are \"all\", \"buffers\", \"wal\", \"general\", \"timing\".",
+ tok);
+ pfree(copy);
+ guc_free(mask);
+ return false;
+ }
+ }
+ pfree(copy);
+
+ *mask = saw_all ? EVS_COLLECT_ALL : accum;
+ if (*mask == 0)
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+}
+
+static void
+evs_collect_assign_hook(const char *newval, void *extra)
+{
+ evs_collect_mask = *((int *) extra);
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0,
+ evs_track_check_hook,
+ evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0,
+ evs_track_relations_check_hook,
+ evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.collect",
+ "Statistics categories to collect.",
+ "Comma- or whitespace-separated list of: "
+ "\"buffers\", \"wal\", \"general\", \"timing\"; "
+ "or \"all\" for every category (default).",
+ &evs_collect, "all",
+ PGC_SUSET, 0,
+ evs_collect_check_hook,
+ evs_collect_assign_hook, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = evs_shmem_request;
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+static void
+evs_shmem_request(void)
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
+ RequestNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME, 1);
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ LWLock *lock = evs_get_track_lock();
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ LWLock *lock = evs_get_track_lock();
+ bool found;
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+/*
+ * Initialize the backend-local tracking hashes and load their contents
+ * from the on-disk file.
+ *
+ * The hashes are per-backend, so no lock is needed to protect them from
+ * other processes; however, another backend may be concurrently rewriting
+ * the track file, so we take a shared lock for the file read.
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+ LWLock *lock;
+ bool need_load;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ lock = evs_get_track_lock();
+
+ if (evs_track_databases_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_databases_hash =
+ hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ if (evs_track_relations_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_relations_hash =
+ hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ need_load = !LWLockHeldByMe(lock);
+ if (need_load)
+ LWLockAcquire(lock, LW_SHARED);
+ PG_TRY();
+ {
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+ }
+ PG_FINALLY();
+ {
+ if (need_load)
+ LWLockRelease(lock);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Load track lists from disk into the backend-local hashes.
+ *
+ * Caller must hold evs_track_lock at least in shared mode, since the file
+ * may be concurrently rewritten by another backend.
+ */
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[MAXPGPATH];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open track file \"%s\": %m", path)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ size_t len = strlen(buf);
+
+ /* Reject unterminated lines (longer than buffer) as corruption. */
+ if (len > 0 && buf[len - 1] != '\n' && !feof(fp))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("line too long in track file \"%s\"", path)));
+
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+
+ if (ferror(fp))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read track file \"%s\": %m", path)));
+ }
+ PG_FINALLY();
+ {
+ FreeFile(fp);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Atomically rewrite the track file. Caller must hold evs_track_lock
+ * in exclusive mode.
+ */
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+ bool failed = false;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s.tmp", path);
+
+ fp = AllocateFile(tmppath, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create track file \"%s\": %m", tmppath)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ if (fputs("[databases]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ if (fprintf(fp, "%u\n", *entry) < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fputs("[relations]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ int rc;
+
+ if (OidIsValid(rel_entry->dboid))
+ rc = fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ rc = fprintf(fp, "0 %u\n", rel_entry->reloid);
+ if (rc < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fflush(fp) != 0)
+ failed = true;
+
+ if (!failed)
+ {
+ int fd = fileno(fp);
+
+ if (fd >= 0 && pg_fsync(fd) != 0)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not fsync track file \"%s\": %m",
+ tmppath)));
+ }
+ }
+ PG_CATCH();
+ {
+ FreeFile(fp);
+ (void) unlink(tmppath);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (FreeFile(fp) != 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close track file \"%s\": %m", tmppath)));
+ failed = true;
+ }
+
+ if (failed)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write track file \"%s\": %m", tmppath)));
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ return;
+ }
+
+ if (durable_rename(tmppath, path, LOG) != 0)
+ {
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ }
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+/*
+ * Mutating track-list entry points: require server-wide privilege, since
+ * the underlying lists steer tracking for every backend.
+ */
+static void
+evs_require_track_privilege(const char *funcname)
+{
+ if (!superuser() && !has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS))
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for function %s", funcname),
+ errhint("Only superusers and members of pg_read_all_stats "
+ "may change the vacuum statistics track list.")));
+}
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid,
+ InvalidOid, NULL);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index ebb7f83d8c5..d7dc0fd07f0 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index b9b03654aad..2a38f9042bb 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 25a85082759..85d721467c0 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 46+ messages in thread
* Re: Vacuum statistics
@ 2026-04-28 05:28 Alena Rybakina <[email protected]>
parent: Alena Rybakina <[email protected]>
0 siblings, 0 replies; 46+ messages in thread
From: Alena Rybakina @ 2026-04-28 05:28 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Amit Kapila <[email protected]>; Jim Nasby <[email protected]>; Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Masahiko Sawada <[email protected]>; Melanie Plageman <[email protected]>; jian he <[email protected]>; Sami Imseih <[email protected]>; vignesh C <[email protected]>; Alexander Korotkov <[email protected]>; Ilia Evdokimov <[email protected]>; Andrey Borodin <[email protected]>; Andrei Zubkov <[email protected]>; Andrei Lepikhov <[email protected]>
On 28.04.2026 05:16, Alena Rybakina wrote:
> Hi, all!
>
> I have updated the core patch that implements the machinery for
> collecting extended vacuum statistics (I didn't touch the first patch
> that is ready for commit, only patches that are related to extension),
> and rebased the ext_vacuum_statistics extension on top of it. The
> split is intentional: the core only gathers metrics and hands them
> out, while the actual storage and SQL-level access to the statistics
> live entirely in the extension. If the extension is not loaded, the
> overhead is essentially zero - we only fill a small struct on the
> stack and do a NULL check on the hook.
>
> What was updated in the core
>
> The core gains the machinery and the hook through which the extension
> receives metrics after each vacuum.
>
> The hook. A new hook has been added in pgstat -
> set_report_vacuum_hook. It is fired once per vacuumed table and once
> per vacuumed index, plus when forming the per-database aggregate. The
> extension registers its handler in _PG_init and by default the hook is
> NULL, so without an extension the core behaves exactly as before.
>
> The set of statistics is the same as before. Common to tables, indexes
> and the database - hits and misses in shared buffers, number of
> dirtied and written pages, WAL volume, buffer read and write times,
> sleep time spent in delay points, total wall-clock vacuum time
> (including I/O and lock waits), counter of emergency anti-wraparound
> vacuums, number of interrupts and removed tuples. Tables additionally
> report frozen tuples, pages marked all-frozen / all-visible in the
> visibility map, number of scanned and removed pages, number of index
> passes, etc. Indexes report freed pages.
>
> The least obvious part of the implementation is subtracting index
> statistics from the table statistics. This is the bit worth
> highlighting. The thing is that indexes are vacuumed before the heap,
> and the buffer and WAL statistics that we capture at the heap level by
> the end of the heap vacuum already include everything that was spent
> on the indexes. If we simply expose the diff of
> pgBufferUsage/pgWalUsage between start and end, the table ends up with
> double-counted pages/WAL: once in its own report, and a second time
> inside the reports of its indexes. This is especially noticeable with
> parallel index vacuum: workers accumulate their usage in the leader
> only after they finish, so without subtraction the heap report would
> receive the combined cost of all workers as a "bonus".
>
> To handle this, as each index finishes vacuuming, its counters are
> accumulated into the state of the current operation, and at the moment
> the heap report is built these sums are subtracted out. As a result,
> the extension receives clean numbers: "this is what was actually spent
> on the table itself", and separately "this is what was actually spent
> on each index". The behaviour is idempotent for both serial and
> parallel vacuum.
>
> The ext_vacuum_statistics extension
>
> The extension registers the hook handler and stores the received data
> through the pgstat custom statistics infrastructure. That is, vacuum
> counters are kept not in the extension's own files, but together with
> the regular cumulative statistics - they survive a restart and are
> reset together with pg_stat_reset_*. Access is provided through three
> views: one for tables, one for indexes, and one with the per-database
> aggregate.
>
> Filtering
>
> This is where the main flexibility lives - the extension does not
> force "collect everything", but lets you choose both what to track and
> which metrics to keep.
>
> By object type. You can limit collection to databases only (without
> per-table detail), to tables only, or collect both. Among tables, you
> can additionally filter system / user / all.
>
> By an explicit list. An alternative to "by type" is a whitelist: you
> turn the corresponding mode on, and the extension starts collecting
> statistics only for the databases and tables that were explicitly
> registered via add_track_database / add_track_relation (with matching
> remove_* for removal). When the lists are off, the type filter is in
> effect; when they are on, only the list applies. This is convenient
> when you are interested in monitoring specific "hot" tables and do not
> want to spend memory on statistics for everything else.
> This list is persisted to disk, and there is one more non-trivial part
> here. List changes are concurrent - multiple sessions may call
> add_track_* simultaneously, plus there is an object-access hook that
> cleans the entry on DROP. To avoid ending up with a torn file, access
> to the list is serialized via a dedicated LWLock tranche (requested
> from a shmem_request_hook), and the file itself is written atomically:
> first into a temporary file, then fflush + pg_fsync + durable_rename.
> All I/O return codes are checked; on error the temporary file is
> removed and the real one is left untouched; PG_TRY/PG_CATCH guarantees
> cleanup on ereport(ERROR). Reading the list takes the same lock in
> shared mode, so a concurrent write cannot tear the load.
>
> By metric category. There is also a GUC that takes a list and turns on
> the categories of interest - buffers, WAL, general counters, timings
> (or all). Unwanted categories are simply skipped on the hook handler
> side and never make it into the pgstat entry, which reduces the
> overhead of the handler itself. This is useful when, for example, only
> timings are needed - in that case the extension does not waste time
> copying the buffer and WAL fields.
>
> Privileges. The add_track_* / remove_track_* functions require
> superuser or pg_read_all_stats. At the SQL level, EXECUTE is revoked
> from PUBLIC and granted only to pg_read_all_stats, so a regular user
> has no access to mutating the list. The views are unrestricted, like
> regular statistics.
>
> What is in the patches
>
> 0002-Machinery-for-grabbing-extended-vacuum-statistics.patch - the
> machinery in the core plus the hook.
> 0003-ext_vacuum_statistics-...patch - the extension itself, filtering,
> views, tests.
>
I noticed CI's complaints during extension installation and fixed it.
--
-----------
Best regards,
Alena Rybakina
Yandex Cloud
From 19f5a39f7e97d3fc2d18415ba2c51ffcd3b32f49 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH 1/3] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
From 3a5e0bd82578d1fea63d6bda229dc4d0b224684e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add infrastructure inside lazy vacuum to gather extended per-vacuum
metrics and expose them to extensions via a new hook. Core itself
does not persist these metrics — that is the job of an extension
(see ext_vacuum_statistics).
Statistics are gathered separately for tables and indexes according
to vacuum phases. The ExtVacReport union and type field distinguish
PGSTAT_EXTVAC_TABLE vs PGSTAT_EXTVAC_INDEX. Heap vacuum stats are
sent to the cumulative statistics system after vacuum has processed
the indexes. Database vacuum statistics aggregate per-table and
per-index statistics within the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
set_report_vacuum_hook (set_report_vacuum_hook_type) -- called
once per vacuumed relation/index with a PgStat_VacuumRelationCounts
payload tagged by ExtVacReportType (PGSTAT_EXTVAC_TABLE / _INDEX /
_DB / _INVALID).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 234 ++++++++++++++++++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 12 +
src/backend/utils/activity/pgstat_relation.c | 24 ++
src/include/commands/vacuum.h | 29 +++
src/include/pgstat.h | 69 ++++++
6 files changed, 367 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 39395aed0d5..e4d4c93d641 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -283,6 +283,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -410,6 +412,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -485,6 +496,166 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -643,7 +814,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_vacuum_min_duration >= 0));
@@ -660,6 +834,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -687,7 +864,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -803,6 +982,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -985,11 +1167,26 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+
pgstat_progress_end_command();
if (instrument)
@@ -2903,6 +3100,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3015,7 +3213,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3033,6 +3235,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3041,6 +3244,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3065,7 +3276,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3084,12 +3299,21 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 99d0db82ed7..a7fb73173f5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2561,6 +2564,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 41cefcfde54..200f12a2d1b 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1076,6 +1076,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -1084,6 +1086,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -1112,6 +1116,13 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1276,6 +1287,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 92e1f60a080..226d7aa06d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -272,6 +272,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 956d9cea36d..a925f7da992 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -21,9 +21,11 @@
#include "catalog/pg_class.h"
#include "catalog/pg_statistic.h"
#include "catalog/pg_type.h"
+#include "executor/instrument.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -354,6 +356,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7db36cf8add..8d934973dc1 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,64 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -703,6 +761,17 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
From 3011a3cfd9ee3d6e4d1c5a12e3d9984f6b6a194e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 28 Apr 2026 03:43:29 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics: extension for extended vacuum
statistics
Introduce a new extension that collects extended per-vacuum
metrics via set_report_vacuum_hook and stores them through pgstat's
custom statistics infrastructure.
Tracking scope is controlled by GUCs:
* vacuum_statistics.enabled -- master switch
* vacuum_statistics.object_types -- databases / relations / all
* vacuum_statistics.track_relations -- system / user / all
* vacuum_statistics.track_{databases,relations}_from_list
-- restrict tracking to objects registered via
add_track_database() / add_track_relation();
removal via remove_track_*() and OAT_DROP hook
* vacuum_statistics.collect -- buffers / wal /
general / timing / all, consulted by ACCUM_IF() to skip
unwanted categories at run time
add_track_* / remove_track_* require superuser or pg_read_all_stats.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 ++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 272 ++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 ++++
.../t/054_vacuum_extending_gucs_test.pl | 279 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1387 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 ++++++
doc/src/sgml/filelist.sgml | 1 +
18 files changed, 3909 insertions(+)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7d91fe77db3..3140f2bf844 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..aa3a9ec9699
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,272 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Track-list mutation requires superuser or pg_read_all_stats; hide the
+-- functions from PUBLIC so the error is also produced for ordinary users
+-- before the C-level privilege check runs.
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) TO pg_read_all_stats;
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT STABLE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..4f8f025c63e
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_frozen_page_marks_cleared(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_visible_page_marks_cleared(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_frozen_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_visible_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..a195249842b
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,279 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra),
+ # Make pending stats visible to subsequent sessions without sleeping.
+ "SELECT pg_stat_force_next_flush();");
+ $node->safe_psql($db, $sql);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session. Assert not only that the
+ # row count is zero, but that the specific counters remain zero: a stray
+ # row with zero counters would otherwise pass a bare COUNT(*)=0 check.
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no rows when disabled');
+
+ my $sums = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_dirtied), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($sums, '0', 'no counters accumulated when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 6: vacuum_statistics.collect - per-category gating
+#
+# With collect='wal' only wal_* counters must advance; buffer, timing, and
+# general categories must stay at zero. With collect='buffers' the inverse
+# holds. Unknown tokens must be rejected by the check-hook.
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.collect' => sub {
+ # wal-only: WAL counters should accumulate, buffers/timing/general should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'wal'"],
+ modify => 1,
+ });
+
+ my $wal = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal, 't', "collect='wal': wal_records accumulated");
+
+ my $other = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0)
+ + COALESCE(SUM(total_time), 0)
+ + COALESCE(SUM(tuples_deleted), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($other, '0',
+ "collect='wal': buffer/timing/general counters not accumulated");
+
+ # buffers-only: buffer counters should advance, WAL should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'buffers'"],
+ modify => 1,
+ });
+
+ my $buf = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($buf, 't', "collect='buffers': buffer counters accumulated");
+
+ my $wal_off = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal_off, '0',
+ "collect='buffers': WAL counters not accumulated");
+
+ # Unknown category must be rejected by the check-hook.
+ my ($ret, $stdout, $stderr) = $node->psql($dbname,
+ "SET vacuum_statistics.collect = 'nope'");
+ isnt($ret, 0, "collect='nope': rejected by check-hook");
+ like($stderr, qr/Unrecognized category "nope"/,
+ "collect='nope': errdetail names the offending token");
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..144b9bcb814
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1387 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_authid.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+#include "utils/tuplestore.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/*
+ * Bit flags for evs_collect_mask. Each category groups counters that can be
+ * accumulated (or skipped) together, letting users reduce overhead at run
+ * time by turning off categories they don't need.
+ */
+#define EVS_COLLECT_BUFFERS 0x1 /* blks_*, blk_*_time */
+#define EVS_COLLECT_WAL 0x2 /* wal_records, wal_fpi, wal_bytes */
+#define EVS_COLLECT_GENERAL 0x4 /* tuples_deleted, pages_*, vm_*,
+ * wraparound_failsafe_count,
+ * interrupts_count */
+#define EVS_COLLECT_TIMING 0x8 /* delay_time, total_time */
+#define EVS_COLLECT_ALL (EVS_COLLECT_BUFFERS | EVS_COLLECT_WAL | \
+ EVS_COLLECT_GENERAL | EVS_COLLECT_TIMING)
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+static char *evs_collect = "all"; /* categories to collect */
+static int evs_collect_mask = EVS_COLLECT_ALL;
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+static shmem_request_hook_type prev_shmem_request_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+static void evs_shmem_request(void);
+
+/* Hash tables for track_databases and track_relations_list (backend-local) */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+/*
+ * Named LWLock tranche protecting the on-disk track file and serializing
+ * backend-local reloads/saves across concurrent backends.
+ */
+#define EVS_TRACK_TRANCHE_NAME "ext_vacuum_statistics_track"
+static LWLock *evs_track_lock = NULL;
+
+static inline LWLock *
+evs_get_track_lock(void)
+{
+ if (evs_track_lock == NULL)
+ evs_track_lock = &GetNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME)->lock;
+ return evs_track_lock;
+}
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/*
+ * Accumulate a single counter only if its category is enabled in
+ * evs_collect_mask. Parentheses around every argument: the macro is invoked
+ * from expression contexts and with expressions as the destination pointer.
+ */
+#define ACCUM_IF(dst, src, field, cat) \
+ do { \
+ if ((evs_collect_mask) & (cat)) \
+ ((dst))->field += ((src))->field; \
+ } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_dirtied, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_written, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_fetched, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_read_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_write_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, delay_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, total_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, wal_records, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_fpi, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_bytes, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wraparound_failsafe_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, interrupts_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, tuples_deleted, EVS_COLLECT_GENERAL);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/*
+ * GUC check hooks: validate the string and compute the bitmask into *extra.
+ * Rejecting unknown values here prevents silent fall-through to "all".
+ */
+static bool
+evs_track_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "databases") == 0)
+ *bits = EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "relations") == 0)
+ *bits = EVS_TRACK_RELATIONS;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"databases\", \"relations\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ evs_track_bits = *((int *) extra);
+}
+
+static bool
+evs_track_relations_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+ else if (strcmp(*newval, "system") == 0)
+ *bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(*newval, "user") == 0)
+ *bits = EVS_FILTER_USER;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"system\", \"user\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ evs_track_relations_bits = *((int *) extra);
+}
+
+/*
+ * Check hook for vacuum_statistics.collect.
+ *
+ * Accepts a comma- or whitespace-separated list of category names
+ * (buffers, wal, general, timing) or the shorthand "all". Computes the
+ * matching bitmask once and stashes it in *extra; the assign hook just
+ * copies it into evs_collect_mask. Unknown tokens are rejected so the
+ * setting cannot silently collapse to the "all" default.
+ */
+static bool
+evs_collect_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *mask;
+ char *copy;
+ char *p;
+ char *tok;
+ int accum = 0;
+ bool saw_all = false;
+
+ if (*newval == NULL)
+ return false;
+
+ mask = (int *) guc_malloc(LOG, sizeof(int));
+ if (!mask)
+ return false;
+
+ /* Empty string means "all", matching the default behavior. */
+ if ((*newval)[0] == '\0')
+ {
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+ }
+
+ copy = pstrdup(*newval);
+ for (p = copy; (tok = strtok(p, " \t,")) != NULL; p = NULL)
+ {
+ if (pg_strcasecmp(tok, "all") == 0)
+ saw_all = true;
+ else if (pg_strcasecmp(tok, "buffers") == 0)
+ accum |= EVS_COLLECT_BUFFERS;
+ else if (pg_strcasecmp(tok, "wal") == 0)
+ accum |= EVS_COLLECT_WAL;
+ else if (pg_strcasecmp(tok, "general") == 0)
+ accum |= EVS_COLLECT_GENERAL;
+ else if (pg_strcasecmp(tok, "timing") == 0)
+ accum |= EVS_COLLECT_TIMING;
+ else
+ {
+ /*
+ * GUC_check_errdetail formats the message immediately, but tok
+ * points into copy; emit the detail first, then free the
+ * scratch buffer so the formatted string is already stashed in
+ * GUC_check_errdetail_string.
+ */
+ GUC_check_errdetail("Unrecognized category \"%s\" in vacuum_statistics.collect; "
+ "allowed values are \"all\", \"buffers\", \"wal\", \"general\", \"timing\".",
+ tok);
+ pfree(copy);
+ guc_free(mask);
+ return false;
+ }
+ }
+ pfree(copy);
+
+ *mask = saw_all ? EVS_COLLECT_ALL : accum;
+ if (*mask == 0)
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+}
+
+static void
+evs_collect_assign_hook(const char *newval, void *extra)
+{
+ evs_collect_mask = *((int *) extra);
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0,
+ evs_track_check_hook,
+ evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0,
+ evs_track_relations_check_hook,
+ evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.collect",
+ "Statistics categories to collect.",
+ "Comma- or whitespace-separated list of: "
+ "\"buffers\", \"wal\", \"general\", \"timing\"; "
+ "or \"all\" for every category (default).",
+ &evs_collect, "all",
+ PGC_SUSET, 0,
+ evs_collect_check_hook,
+ evs_collect_assign_hook, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = evs_shmem_request;
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+static void
+evs_shmem_request(void)
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
+ RequestNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME, 1);
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ LWLock *lock = evs_get_track_lock();
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ LWLock *lock = evs_get_track_lock();
+ bool found;
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+/*
+ * Initialize the backend-local tracking hashes and load their contents
+ * from the on-disk file.
+ *
+ * The hashes are per-backend, so no lock is needed to protect them from
+ * other processes; however, another backend may be concurrently rewriting
+ * the track file, so we take a shared lock for the file read.
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+ LWLock *lock;
+ bool need_load;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ lock = evs_get_track_lock();
+
+ if (evs_track_databases_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_databases_hash =
+ hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ if (evs_track_relations_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_relations_hash =
+ hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ need_load = !LWLockHeldByMe(lock);
+ if (need_load)
+ LWLockAcquire(lock, LW_SHARED);
+ PG_TRY();
+ {
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+ }
+ PG_FINALLY();
+ {
+ if (need_load)
+ LWLockRelease(lock);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Load track lists from disk into the backend-local hashes.
+ *
+ * Caller must hold evs_track_lock at least in shared mode, since the file
+ * may be concurrently rewritten by another backend.
+ */
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[MAXPGPATH];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open track file \"%s\": %m", path)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ size_t len = strlen(buf);
+
+ /* Reject unterminated lines (longer than buffer) as corruption. */
+ if (len > 0 && buf[len - 1] != '\n' && !feof(fp))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("line too long in track file \"%s\"", path)));
+
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+
+ if (ferror(fp))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read track file \"%s\": %m", path)));
+ }
+ PG_FINALLY();
+ {
+ FreeFile(fp);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Atomically rewrite the track file. Caller must hold evs_track_lock
+ * in exclusive mode.
+ */
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+ bool failed = false;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s.tmp", path);
+
+ fp = AllocateFile(tmppath, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create track file \"%s\": %m", tmppath)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ if (fputs("[databases]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ if (fprintf(fp, "%u\n", *entry) < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fputs("[relations]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ int rc;
+
+ if (OidIsValid(rel_entry->dboid))
+ rc = fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ rc = fprintf(fp, "0 %u\n", rel_entry->reloid);
+ if (rc < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fflush(fp) != 0)
+ failed = true;
+
+ if (!failed)
+ {
+ int fd = fileno(fp);
+
+ if (fd >= 0 && pg_fsync(fd) != 0)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not fsync track file \"%s\": %m",
+ tmppath)));
+ }
+ }
+ PG_CATCH();
+ {
+ FreeFile(fp);
+ (void) unlink(tmppath);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (FreeFile(fp) != 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close track file \"%s\": %m", tmppath)));
+ failed = true;
+ }
+
+ if (failed)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write track file \"%s\": %m", tmppath)));
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ return;
+ }
+
+ if (durable_rename(tmppath, path, LOG) != 0)
+ {
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ }
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+/*
+ * Mutating track-list entry points: require server-wide privilege, since
+ * the underlying lists steer tracking for every backend.
+ */
+static void
+evs_require_track_privilege(const char *funcname)
+{
+ if (!superuser() && !has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS))
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for function %s", funcname),
+ errhint("Only superusers and members of pg_read_all_stats "
+ "may change the vacuum statistics track list.")));
+}
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base PG_USED_FOR_ASSERTS_ONLY = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid,
+ InvalidOid, NULL);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index ebb7f83d8c5..d7dc0fd07f0 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index b9b03654aad..2a38f9042bb 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 25a85082759..85d721467c0 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v39-0001-Track-table-VM-stability.patch (21.7K, 3-v39-0001-Track-table-VM-stability.patch)
download | inline diff:
From 19f5a39f7e97d3fc2d18415ba2c51ffcd3b32f49 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 30 Mar 2026 09:07:24 +0300
Subject: [PATCH 1/3] Track table VM stability.
Add rev_all_visible_pages and rev_all_frozen_pages counters to
pg_stat_all_tables tracking the number of times the all-visible and
all-frozen bits are cleared in the visibility map. These bits are cleared by
backend processes during regular DML operations. Hence, the counters are placed
in table statistic entry.
A high rev_all_visible_pages rate relative to DML volume indicates
that modifications are scattered across previously-clean pages rather
than concentrated on already-dirty ones, causing index-only scans to
fall back to heap fetches. A high rev_all_frozen_pages rate indicates
that vacuum's freezing work is being frequently undone by concurrent
DML.
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
Jian He <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>,
Andrey Borodin <[email protected]>
---
doc/src/sgml/monitoring.sgml | 32 +++
src/backend/access/heap/visibilitymap.c | 10 +
src/backend/catalog/system_views.sql | 4 +-
src/backend/utils/activity/pgstat_relation.c | 2 +
src/backend/utils/adt/pgstatfuncs.c | 6 +
src/include/catalog/pg_proc.dat | 10 +
src/include/pgstat.h | 17 +-
.../expected/vacuum-extending-freeze.out | 185 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/vacuum-extending-freeze.spec | 117 +++++++++++
src/test/regress/expected/rules.out | 12 +-
11 files changed, 391 insertions(+), 5 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extending-freeze.out
create mode 100644 src/test/isolation/specs/vacuum-extending-freeze.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 08d5b824552..3467abf6d8a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4377,6 +4377,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>visible_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-visible mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-visible mark of a heap page is
+ cleared whenever a backend process modifies a page that was
+ previously marked all-visible by vacuum activity (whether manual
+ <command>VACUUM</command> or autovacuum). The page must then be
+ processed again by vacuum on a subsequent run. A high rate of
+ change in this counter means that vacuum has to repeatedly
+ re-process pages of this table.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>frozen_page_marks_cleared</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the all-frozen mark in the
+ <link linkend="storage-vm">visibility map</link> was cleared for
+ pages of this table. The all-frozen mark of a heap page is cleared
+ whenever a backend process modifies a page that was previously
+ marked all-frozen by vacuum activity (manual <command>VACUUM</command>
+ or autovacuum). The page must then be processed again by vacuum on
+ the next freeze run for this table.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>last_vacuum</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 4fd470702aa..f055ec3819c 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -102,6 +102,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
@@ -173,6 +174,15 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Track how often all-visible or all-frozen bits are cleared in the
+ * visibility map.
+ */
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_VISIBLE) << mapOffset))
+ pgstat_count_visible_page_marks_cleared(rel);
+ if (map[mapByte] & ((flags & VISIBILITYMAP_ALL_FROZEN) << mapOffset))
+ pgstat_count_frozen_page_marks_cleared(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 73a1c1c4670..71e993c8783 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -747,7 +747,9 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_total_autovacuum_time(C.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(C.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(C.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(C.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(C.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(C.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(C.oid) AS frozen_page_marks_cleared
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index b2ca28f83ba..92e1f60a080 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -881,6 +881,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->visible_page_marks_cleared += lstats->counts.visible_page_marks_cleared;
+ tabentry->frozen_page_marks_cleared += lstats->counts.frozen_page_marks_cleared;
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 1408de387ea..b6f064338fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -108,6 +108,12 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated)
/* pg_stat_get_vacuum_count */
PG_STAT_GET_RELENTRY_INT64(vacuum_count)
+/* pg_stat_get_visible_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(visible_page_marks_cleared)
+
+/* pg_stat_get_frozen_page_marks_cleared */
+PG_STAT_GET_RELENTRY_INT64(frozen_page_marks_cleared)
+
#define PG_STAT_GET_RELENTRY_FLOAT8(stat) \
Datum \
CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa9ae79082b..f8241268017 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12769,4 +12769,14 @@
proname => 'hashoid8extended', prorettype => 'int8',
proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' },
+{ oid => '8002',
+ descr => 'statistics: number of times the all-visible marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_visible_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_visible_page_marks_cleared' },
+{ oid => '8003',
+ descr => 'statistics: number of times the all-frozen marks in the visibility map were cleared for pages of this table',
+ proname => 'pg_stat_get_frozen_page_marks_cleared', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_frozen_page_marks_cleared' },
]
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index dfa2e837638..7db36cf8add 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -160,6 +160,8 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
} PgStat_TableCounts;
/* ----------
@@ -218,7 +220,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -469,6 +471,8 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+ PgStat_Counter visible_page_marks_cleared;
+ PgStat_Counter frozen_page_marks_cleared;
TimestampTz last_vacuum_time; /* user initiated vacuum */
PgStat_Counter vacuum_count;
@@ -749,6 +753,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* count revocations of all-visible and all-frozen marks in visibility map */
+#define pgstat_count_visible_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.visible_page_marks_cleared++; \
+ } while (0)
+#define pgstat_count_frozen_page_marks_cleared(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.frozen_page_marks_cleared++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
diff --git a/src/test/isolation/expected/vacuum-extending-freeze.out b/src/test/isolation/expected/vacuum-extending-freeze.out
new file mode 100644
index 00000000000..994a8df56df
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending-freeze.out
@@ -0,0 +1,185 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_vacuum_freeze s1_get_set_vm_flags_stats s1_update_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s2_vacuum_freeze s1_select_from_index s2_delete_from_table s1_get_cleared_vm_flags_stats s2_vacuum_freeze s1_get_set_vm_flags_stats s1_commit s1_get_cleared_vm_flags_stats
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s1_update_table:
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+t |t
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_select_from_index:
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+
+count
+-----
+ 3000
+(1 row)
+
+step s2_delete_from_table:
+ DELETE FROM vestat WHERE x > 4930;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+f |f
+(1 row)
+
+step s2_vacuum_freeze:
+ VACUUM FREEZE vestat;
+
+step s1_get_set_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+relallfrozen|relallvisible
+------------+-------------
+f |f
+(1 row)
+
+step s1_commit:
+ COMMIT;
+
+step s1_get_cleared_vm_flags_stats:
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+
+pg_stat_force_next_flush
+------------------------
+
+(1 row)
+
+visible_page_marks_cleared|frozen_page_marks_cleared
+--------------------------+-------------------------
+t |t
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 1578ba191c8..91ffc57ebd4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -126,3 +126,4 @@ test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
test: for-portion-of
+test: vacuum-extending-freeze
diff --git a/src/test/isolation/specs/vacuum-extending-freeze.spec b/src/test/isolation/specs/vacuum-extending-freeze.spec
new file mode 100644
index 00000000000..17c204e2326
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending-freeze.spec
@@ -0,0 +1,117 @@
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across VACUUM and backend operations.
+# In addition, the test provides a scenario where one process holds a
+# transaction open while another process deletes tuples. We expect that
+# a backend clears the all-frozen and all-visible flags, which were set
+# by VACUUM earlier, only after the committing transaction makes the
+# deletions visible.
+
+setup
+{
+ CREATE TABLE vestat (x int, y int)
+ WITH (autovacuum_enabled = off, fillfactor = 70);
+
+ INSERT INTO vestat
+ SELECT i, i FROM generate_series(1, 5000) AS g(i);
+
+ CREATE INDEX vestat_idx ON vestat (x);
+
+ CREATE TABLE stats_state (frozen_flag_count int, all_visibile_flag_count int,
+ cleared_frozen_flag_count int, cleared_all_visibile_flag_count int);
+ INSERT INTO stats_state VALUES (0,0,0,0);
+ ANALYZE vestat;
+
+ -- Ensure stats are flushed before starting the scenario
+ SELECT pg_stat_force_next_flush();
+}
+
+teardown
+{
+ DROP TABLE IF EXISTS vestat;
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+
+}
+
+session s1
+
+step s1_get_set_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT c.relallfrozen > frozen_flag_count as relallfrozen, c.relallvisible > all_visibile_flag_count as relallvisible
+ FROM pg_class c, stats_state
+ WHERE c.relname = 'vestat';
+
+ UPDATE stats_state
+ SET frozen_flag_count = c.relallfrozen,
+ all_visibile_flag_count = c.relallvisible
+ FROM pg_class c
+ WHERE c.relname = 'vestat';
+}
+
+step s1_get_cleared_vm_flags_stats
+{
+ SELECT pg_stat_force_next_flush();
+
+ SELECT v.visible_page_marks_cleared > cleared_all_visibile_flag_count as visible_page_marks_cleared,
+ v.frozen_page_marks_cleared > cleared_frozen_flag_count as frozen_page_marks_cleared
+ FROM pg_stat_all_tables v, stats_state
+ WHERE v.relname = 'vestat';
+
+ UPDATE stats_state
+ SET cleared_all_visibile_flag_count = v.visible_page_marks_cleared,
+ cleared_frozen_flag_count = v.frozen_page_marks_cleared
+ FROM pg_stat_all_tables v
+ WHERE v.relname = 'vestat';
+}
+
+step s1_select_from_index
+{
+ BEGIN;
+ SELECT count(x) FROM vestat WHERE x > 2000;
+}
+
+step s1_commit
+{
+ COMMIT;
+}
+
+session s2
+setup
+{
+ -- Configure aggressive freezing vacuum behavior
+ SET vacuum_freeze_min_age = 0;
+ SET vacuum_freeze_table_age = 0;
+}
+step s2_delete_from_table
+{
+ DELETE FROM vestat WHERE x > 4930;
+}
+step s2_vacuum_freeze
+{
+ VACUUM FREEZE vestat;
+}
+
+step s1_update_table
+{
+ UPDATE vestat SET x = x + 1001 where x >= 2500;
+ SELECT pg_stat_force_next_flush();
+}
+
+permutation
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_update_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s2_vacuum_freeze
+ s1_select_from_index
+ s2_delete_from_table
+ s1_get_cleared_vm_flags_stats
+ s2_vacuum_freeze
+ s1_get_set_vm_flags_stats
+ s1_commit
+ s1_get_cleared_vm_flags_stats
\ No newline at end of file
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a65a5bf0c4f..096e4f763f3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1846,7 +1846,9 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_total_autovacuum_time(c.oid) AS total_autovacuum_time,
pg_stat_get_total_analyze_time(c.oid) AS total_analyze_time,
pg_stat_get_total_autoanalyze_time(c.oid) AS total_autoanalyze_time,
- pg_stat_get_stat_reset_time(c.oid) AS stats_reset
+ pg_stat_get_stat_reset_time(c.oid) AS stats_reset,
+ pg_stat_get_visible_page_marks_cleared(c.oid) AS visible_page_marks_cleared,
+ pg_stat_get_frozen_page_marks_cleared(c.oid) AS frozen_page_marks_cleared
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace)))
@@ -2357,7 +2359,9 @@ pg_stat_sys_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
pg_stat_user_functions| SELECT p.oid AS funcid,
@@ -2412,7 +2416,9 @@ pg_stat_user_tables| SELECT relid,
total_autovacuum_time,
total_analyze_time,
total_autoanalyze_time,
- stats_reset
+ stats_reset,
+ visible_page_marks_cleared,
+ frozen_page_marks_cleared
FROM pg_stat_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
pg_stat_wal| SELECT wal_records,
--
2.39.5 (Apple Git-154)
[text/plain] v39-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch (25.0K, 4-v39-0002-Machinery-for-grabbing-extended-vacuum-statistics.patch)
download | inline diff:
From 3a5e0bd82578d1fea63d6bda229dc4d0b224684e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Mon, 2 Mar 2026 23:09:32 +0300
Subject: [PATCH 2/3] Machinery for grabbing extended vacuum statistics.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add infrastructure inside lazy vacuum to gather extended per-vacuum
metrics and expose them to extensions via a new hook. Core itself
does not persist these metrics — that is the job of an extension
(see ext_vacuum_statistics).
Statistics are gathered separately for tables and indexes according
to vacuum phases. The ExtVacReport union and type field distinguish
PGSTAT_EXTVAC_TABLE vs PGSTAT_EXTVAC_INDEX. Heap vacuum stats are
sent to the cumulative statistics system after vacuum has processed
the indexes. Database vacuum statistics aggregate per-table and
per-index statistics within the database.
Common for tables, indexes, and database: total_blks_hit, total_blks_read
and total_blks_dirtied are the number of hit, miss and dirtied pages
in shared buffers during a vacuum operation. total_blks_dirtied counts
only pages dirtied by this vacuum. blk_read_time and blk_write_time
track access and flush time for buffer pages; blk_write_time can stay
zero if no flushes occurred. total_time is wall-clock time from start
to finish, including idle time (I/O and lock waits). delay_time is
total vacuum sleep time in vacuum delay points.
Both table and index report tuples_deleted (tuples removed by the vacuum),
pages_removed (pages by which relation storage was reduced) and
pages_deleted (freed pages; file size may remain unchanged). These are
independent of WAL and buffer stats and are not summed at the database
level.
Table only: pages_frozen (pages marked all-frozen in the visibility map),
pages_all_visible (pages marked all-visible in the visibility map),
wraparound_failsafe_count (number of urgent anti-wraparound vacuums).
Table and database share wraparound_failsafe (count of urgent anti-wraparound
cleanups). Database only: errors (number of error-level errors caught
during vacuum).
set_report_vacuum_hook (set_report_vacuum_hook_type) -- called
once per vacuumed relation/index with a PgStat_VacuumRelationCounts
payload tagged by ExtVacReportType (PGSTAT_EXTVAC_TABLE / _INDEX /
_DB / _INVALID).
Authors: Alena Rybakina <[email protected]>,
Andrei Lepikhov <[email protected]>,
Andrei Zubkov <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>,
Masahiko Sawada <[email protected]>,
Ilia Evdokimov <[email protected]>,
jian he <[email protected]>,
Kirill Reshke <[email protected]>,
Alexander Korotkov <[email protected]>,
Jim Nasby <[email protected]>,
Sami Imseih <[email protected]>,
Karina Litskevich <[email protected]>
---
src/backend/access/heap/vacuumlazy.c | 234 ++++++++++++++++++-
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 12 +
src/backend/utils/activity/pgstat_relation.c | 24 ++
src/include/commands/vacuum.h | 29 +++
src/include/pgstat.h | 69 ++++++
6 files changed, 367 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 39395aed0d5..e4d4c93d641 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -283,6 +283,8 @@ typedef struct LVRelState
/* Error reporting state */
char *dbname;
char *relnamespace;
+ Oid reloid;
+ Oid indoid;
char *relname;
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
@@ -410,6 +412,15 @@ typedef struct LVRelState
* been permanently disabled.
*/
BlockNumber eager_scan_remaining_fails;
+
+ int32 wraparound_failsafe_count; /* # of emergency vacuums for
+ * anti-wraparound */
+
+ /*
+ * We need to accumulate index statistics for later subtraction from heap
+ * stats.
+ */
+ PgStat_VacuumRelationCounts extVacReportIdx;
} LVRelState;
@@ -485,6 +496,166 @@ static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* Extended vacuum statistics functions */
+
+/*
+ * extvac_stats_start - Save cut-off values before start of relation processing.
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters * counters)
+{
+ memset(counters, 0, sizeof(LVExtStatCounters));
+ counters->starttime = GetCurrentTimestamp();
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_end - Finish extended vacuum statistic gathering and form report.
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters * counters,
+ PgStat_CommonCounts * report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+
+ memset(report, 0, sizeof(PgStat_CommonCounts));
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->starttime, endtime, &secs, &usecs);
+
+ report->total_blks_read = bufusage.local_blks_read + bufusage.shared_blks_read;
+ report->total_blks_hit = bufusage.local_blks_hit + bufusage.shared_blks_hit;
+ report->total_blks_dirtied = bufusage.local_blks_dirtied + bufusage.shared_blks_dirtied;
+ report->total_blks_written = bufusage.shared_blks_written;
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time) +
+ INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+ report->total_time = secs * 1000.0 + usecs / 1000.0;
+
+ if (rel->pgstat_info && pgstat_track_counts)
+ {
+ report->blks_fetched = rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit = rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+ }
+}
+
+/*
+ * extvac_stats_start_idx - Start extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = 0;
+ counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+
+/*
+ * extvac_stats_end_idx - Finish extended vacuum statistic gathering for index.
+ */
+void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx * counters, PgStat_VacuumRelationCounts * report)
+{
+ memset(report, 0, sizeof(PgStat_VacuumRelationCounts));
+ extvac_stats_end(rel, &counters->common, &report->common);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ report->common.tuples_deleted = stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted = stats->pages_deleted - counters->pages_deleted;
+ }
+}
+
+/*
+ * Accumulate index stats into vacrel for later subtraction from heap stats.
+ * It needs to prevent double-counting of stats for heaps that
+ * include indexes because indexes are vacuumed before the heap.
+ * We need to be careful with buffer usage and wal usage during parallel vacuum
+ * because they are accumulated summarly for all indexes at once by leader after
+ * all workers have finished.
+ */
+static void
+accumulate_idxs_vacuum_statistics(LVRelState *vacrel,
+ PgStat_VacuumRelationCounts * extVacIdxStats)
+{
+ vacrel->extVacReportIdx.common.blk_read_time += extVacIdxStats->common.blk_read_time;
+ vacrel->extVacReportIdx.common.blk_write_time += extVacIdxStats->common.blk_write_time;
+ vacrel->extVacReportIdx.common.total_blks_dirtied += extVacIdxStats->common.total_blks_dirtied;
+ vacrel->extVacReportIdx.common.total_blks_hit += extVacIdxStats->common.total_blks_hit;
+ vacrel->extVacReportIdx.common.total_blks_read += extVacIdxStats->common.total_blks_read;
+ vacrel->extVacReportIdx.common.total_blks_written += extVacIdxStats->common.total_blks_written;
+ vacrel->extVacReportIdx.common.wal_bytes += extVacIdxStats->common.wal_bytes;
+ vacrel->extVacReportIdx.common.wal_fpi += extVacIdxStats->common.wal_fpi;
+ vacrel->extVacReportIdx.common.wal_records += extVacIdxStats->common.wal_records;
+ vacrel->extVacReportIdx.common.delay_time += extVacIdxStats->common.delay_time;
+ vacrel->extVacReportIdx.common.total_time += extVacIdxStats->common.total_time;
+}
+
+/* Build heap-specific extended stats */
+static void
+accumulate_heap_vacuum_statistics(LVRelState *vacrel, PgStat_VacuumRelationCounts * extVacStats)
+{
+ extVacStats->type = PGSTAT_EXTVAC_TABLE;
+ extVacStats->table.pages_scanned = vacrel->scanned_pages;
+ extVacStats->table.pages_removed = vacrel->removed_pages;
+ extVacStats->table.vm_new_frozen_pages = vacrel->new_all_frozen_pages;
+ extVacStats->table.vm_new_visible_pages = vacrel->new_all_visible_pages;
+ extVacStats->table.vm_new_visible_frozen_pages = vacrel->new_all_visible_all_frozen_pages;
+ extVacStats->common.tuples_deleted = vacrel->tuples_deleted;
+ extVacStats->table.tuples_frozen = vacrel->tuples_frozen;
+ extVacStats->table.recently_dead_tuples = vacrel->recently_dead_tuples;
+ extVacStats->table.missed_dead_tuples = vacrel->missed_dead_tuples;
+ extVacStats->table.missed_dead_pages = vacrel->missed_dead_pages;
+ extVacStats->table.index_vacuum_count = vacrel->num_index_scans;
+ extVacStats->common.wraparound_failsafe_count = vacrel->wraparound_failsafe_count;
+
+ /* Hook is invoked from pgstat_report_vacuum() when extstats is passed */
+
+ /* Subtract index stats from heap to avoid double-counting */
+ extVacStats->common.blk_read_time -= vacrel->extVacReportIdx.common.blk_read_time;
+ extVacStats->common.blk_write_time -= vacrel->extVacReportIdx.common.blk_write_time;
+ extVacStats->common.total_blks_dirtied -= vacrel->extVacReportIdx.common.total_blks_dirtied;
+ extVacStats->common.total_blks_hit -= vacrel->extVacReportIdx.common.total_blks_hit;
+ extVacStats->common.total_blks_read -= vacrel->extVacReportIdx.common.total_blks_read;
+ extVacStats->common.total_blks_written -= vacrel->extVacReportIdx.common.total_blks_written;
+ extVacStats->common.wal_bytes -= vacrel->extVacReportIdx.common.wal_bytes;
+ extVacStats->common.wal_fpi -= vacrel->extVacReportIdx.common.wal_fpi;
+ extVacStats->common.wal_records -= vacrel->extVacReportIdx.common.wal_records;
+ extVacStats->common.total_time -= vacrel->extVacReportIdx.common.total_time;
+ extVacStats->common.delay_time -= vacrel->extVacReportIdx.common.delay_time;
+}
/*
* Helper to set up the eager scanning state for vacuuming a single relation.
@@ -643,7 +814,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
ErrorContextCallback errcallback;
char **indnames = NULL;
Size dead_items_max_bytes = 0;
+ LVExtStatCounters extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ memset(&extVacReport, 0, sizeof(extVacReport));
verbose = (params->options & VACOPT_VERBOSE) != 0;
instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_vacuum_min_duration >= 0));
@@ -660,6 +834,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
/* Used for instrumentation and stats report */
starttime = GetCurrentTimestamp();
+ if (set_report_vacuum_hook)
+ extvac_stats_start(rel, &extVacCounters);
+
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
if (AmAutoVacuumWorkerProcess())
@@ -687,7 +864,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
+ memset(&vacrel->extVacReportIdx, 0, sizeof(vacrel->extVacReportIdx));
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
errcallback.callback = vacuum_error_callback;
@@ -803,6 +982,9 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
vacrel->vistest = GlobalVisTestFor(rel);
+ /* Initialize wraparound failsafe count for extended vacuum stats */
+ vacrel->wraparound_failsafe_count = 0;
+
/* Initialize state used to track oldest extant XID/MXID */
vacrel->NewRelfrozenXid = vacrel->cutoffs.OldestXmin;
vacrel->NewRelminMxid = vacrel->cutoffs.OldestMxact;
@@ -985,11 +1167,26 @@ heap_vacuum_rel(Relation rel, const VacuumParams *params,
* soon in cases where the failsafe prevented significant amounts of heap
* vacuuming.
*/
- pgstat_report_vacuum(rel,
- Max(vacrel->new_live_tuples, 0),
- vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples,
- starttime);
+ if (set_report_vacuum_hook)
+ {
+ extvac_stats_end(rel, &extVacCounters, &extVacReport.common);
+ accumulate_heap_vacuum_statistics(vacrel, &extVacReport);
+
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ &extVacReport);
+ }
+ else
+ pgstat_report_vacuum_ext(rel,
+ Max(vacrel->new_live_tuples, 0),
+ vacrel->recently_dead_tuples +
+ vacrel->missed_dead_tuples,
+ starttime,
+ NULL);
+
pgstat_progress_end_command();
if (instrument)
@@ -2903,6 +3100,7 @@ lazy_check_wraparound_failsafe(LVRelState *vacrel)
int64 progress_val[3] = {0, 0, PROGRESS_VACUUM_MODE_FAILSAFE};
VacuumFailsafeActive = true;
+ vacrel->wraparound_failsafe_count++;
/*
* Abandon use of a buffer access strategy to allow use of all of
@@ -3015,7 +3213,11 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3033,6 +3235,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -3041,6 +3244,14 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, vacrel->dead_items,
vacrel->dead_items_info);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3065,7 +3276,11 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
ivinfo.analyze_only = false;
@@ -3084,12 +3299,21 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ accumulate_idxs_vacuum_statistics(vacrel, &extVacReport);
+ }
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 99d0db82ed7..a7fb73173f5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -118,6 +118,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time (msec). */
+double VacuumDelayTime = 0;
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2561,6 +2564,7 @@ vacuum_delay_point(bool is_analyze)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 41cefcfde54..200f12a2d1b 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1076,6 +1076,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
IndexBulkDeleteResult *istat = NULL;
IndexBulkDeleteResult *istat_res;
IndexVacuumInfo ivinfo;
+ LVExtStatCountersIdx extVacCounters;
+ PgStat_VacuumRelationCounts extVacReport;
/*
* Update the pointer to the corresponding bulk-deletion result if someone
@@ -1084,6 +1086,8 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
if (indstats->istat_updated)
istat = &(indstats->istat);
+ if (set_report_vacuum_hook)
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = pvs->heaprel;
ivinfo.analyze_only = false;
@@ -1112,6 +1116,13 @@ parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
RelationGetRelationName(indrel));
}
+ if (set_report_vacuum_hook)
+ {
+ memset(&extVacReport, 0, sizeof(extVacReport));
+ extvac_stats_end_idx(indrel, istat_res, &extVacCounters, &extVacReport);
+ pgstat_report_vacuum_ext(indrel, -1, -1, 0, &extVacReport);
+ }
+
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
* amvacuumcleanup to the DSM segment if it's the first cycle because they
@@ -1276,6 +1287,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
VacuumUpdateCosts();
VacuumCostBalance = 0;
+ VacuumDelayTime = 0;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 92e1f60a080..226d7aa06d5 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -272,6 +272,30 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
(void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
+/*
+ * Hook for extensions to receive extended vacuum statistics.
+ * NULL when no extension has registered.
+ */
+set_report_vacuum_hook_type set_report_vacuum_hook = NULL;
+
+/*
+ * Report extended vacuum statistics to extensions via set_report_vacuum_hook.
+ * When livetuples/deadtuples/starttime are provided (heap case), also calls
+ * pgstat_report_vacuum. For indexes, pass -1, -1, 0 to skip pgstat_report_vacuum.
+ */
+void
+pgstat_report_vacuum_ext(Relation rel, PgStat_Counter livetuples,
+ PgStat_Counter deadtuples, TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats)
+{
+ pgstat_report_vacuum(rel, livetuples, deadtuples, starttime);
+
+ if (extstats != NULL && set_report_vacuum_hook)
+ (*set_report_vacuum_hook) (RelationGetRelid(rel),
+ rel->rd_rel->relisshared,
+ extstats);
+}
+
/*
* Report that the table was just analyzed and flush IO statistics.
*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 956d9cea36d..a925f7da992 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -21,9 +21,11 @@
#include "catalog/pg_class.h"
#include "catalog/pg_statistic.h"
#include "catalog/pg_type.h"
+#include "executor/instrument.h"
#include "parser/parse_node.h"
#include "storage/buf.h"
#include "utils/relcache.h"
+#include "pgstat.h"
/*
* Flags for amparallelvacuumoptions to control the participation of bulkdelete
@@ -354,6 +356,33 @@ extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+/* Cumulative storage to report total vacuum delay time (msec). */
+extern PGDLLIMPORT double VacuumDelayTime;
+
+/* Counters for extended vacuum statistics gathering */
+typedef struct LVExtStatCounters
+{
+ TimestampTz starttime;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
+
+extern void extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters);
+extern void extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters,
+ PgStat_VacuumRelationCounts *report);
+
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
extern PGDLLIMPORT int vacuum_cost_limit;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7db36cf8add..8d934973dc1 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -93,6 +93,64 @@ typedef struct PgStat_FunctionCounts
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
+/*
+ * Extended vacuum statistics - passed to extensions via set_report_vacuum_hook.
+ * Type of entry: table (heap), index, or database aggregate.
+ */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_TABLE = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+typedef struct PgStat_CommonCounts
+{
+ int64 total_blks_read;
+ int64 total_blks_hit;
+ int64 total_blks_dirtied;
+ int64 total_blks_written;
+ int64 blks_fetched;
+ int64 blks_hit;
+ int64 wal_records;
+ int64 wal_fpi;
+ uint64 wal_bytes;
+ double blk_read_time;
+ double blk_write_time;
+ double delay_time;
+ double total_time;
+ int32 wraparound_failsafe_count;
+ int32 interrupts_count;
+ int64 tuples_deleted;
+} PgStat_CommonCounts;
+
+typedef struct PgStat_VacuumRelationCounts
+{
+ PgStat_CommonCounts common;
+ ExtVacReportType type;
+ union
+ {
+ struct
+ {
+ int64 tuples_frozen;
+ int64 recently_dead_tuples;
+ int64 missed_dead_tuples;
+ int64 pages_scanned;
+ int64 pages_removed;
+ int64 vm_new_frozen_pages;
+ int64 vm_new_visible_pages;
+ int64 vm_new_visible_frozen_pages;
+ int64 missed_dead_pages;
+ int64 index_vacuum_count;
+ } table;
+ struct
+ {
+ int64 pages_deleted;
+ } index;
+ };
+} PgStat_VacuumRelationCounts;
+
typedef struct PgStat_FunctionCallUsage
{
/* Link to function's hashtable entry (must still be there at exit!) */
@@ -703,6 +761,17 @@ extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
PgStat_Counter deadtuples,
TimestampTz starttime);
+
+extern void pgstat_report_vacuum_ext(Relation rel,
+ PgStat_Counter livetuples,
+ PgStat_Counter deadtuples,
+ TimestampTz starttime,
+ PgStat_VacuumRelationCounts * extstats);
+
+/* Hook for extensions to receive extended vacuum statistics */
+typedef void (*set_report_vacuum_hook_type) (Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+extern PGDLLIMPORT set_report_vacuum_hook_type set_report_vacuum_hook;
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter, TimestampTz starttime);
--
2.39.5 (Apple Git-154)
[text/plain] v39-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch (145.2K, 5-v39-0003-ext_vacuum_statistics-extension-for-extended-vacuum-.patch)
download | inline diff:
From 3011a3cfd9ee3d6e4d1c5a12e3d9984f6b6a194e Mon Sep 17 00:00:00 2001
From: Alena Rybakina <[email protected]>
Date: Tue, 28 Apr 2026 03:43:29 +0300
Subject: [PATCH 3/3] ext_vacuum_statistics: extension for extended vacuum
statistics
Introduce a new extension that collects extended per-vacuum
metrics via set_report_vacuum_hook and stores them through pgstat's
custom statistics infrastructure.
Tracking scope is controlled by GUCs:
* vacuum_statistics.enabled -- master switch
* vacuum_statistics.object_types -- databases / relations / all
* vacuum_statistics.track_relations -- system / user / all
* vacuum_statistics.track_{databases,relations}_from_list
-- restrict tracking to objects registered via
add_track_database() / add_track_relation();
removal via remove_track_*() and OAT_DROP hook
* vacuum_statistics.collect -- buffers / wal /
general / timing / all, consulted by ACCUM_IF() to skip
unwanted categories at run time
add_track_* / remove_track_* require superuser or pg_read_all_stats.
---
contrib/Makefile | 1 +
contrib/ext_vacuum_statistics/Makefile | 24 +
contrib/ext_vacuum_statistics/README.md | 165 ++
.../expected/ext_vacuum_statistics.out | 52 +
.../vacuum-extending-in-repetable-read.out | 52 +
.../ext_vacuum_statistics--1.0.sql | 272 ++++
.../ext_vacuum_statistics.conf | 2 +
.../ext_vacuum_statistics.control | 5 +
contrib/ext_vacuum_statistics/meson.build | 41 +
.../vacuum-extending-in-repetable-read.spec | 59 +
.../t/052_vacuum_extending_basic_test.pl | 780 +++++++++
.../t/053_vacuum_extending_freeze_test.pl | 285 ++++
.../t/054_vacuum_extending_gucs_test.pl | 279 ++++
.../ext_vacuum_statistics/vacuum_statistics.c | 1387 +++++++++++++++++
contrib/meson.build | 1 +
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/extvacuumstatistics.sgml | 502 ++++++
doc/src/sgml/filelist.sgml | 1 +
18 files changed, 3909 insertions(+)
create mode 100644 contrib/ext_vacuum_statistics/Makefile
create mode 100644 contrib/ext_vacuum_statistics/README.md
create mode 100644 contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
create mode 100644 contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
create mode 100644 contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
create mode 100644 contrib/ext_vacuum_statistics/meson.build
create mode 100644 contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
create mode 100644 contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
create mode 100644 contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
create mode 100644 contrib/ext_vacuum_statistics/vacuum_statistics.c
create mode 100644 doc/src/sgml/extvacuumstatistics.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7d91fe77db3..3140f2bf844 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -19,6 +19,7 @@ SUBDIRS = \
dict_int \
dict_xsyn \
earthdistance \
+ ext_vacuum_statistics \
file_fdw \
fuzzystrmatch \
hstore \
diff --git a/contrib/ext_vacuum_statistics/Makefile b/contrib/ext_vacuum_statistics/Makefile
new file mode 100644
index 00000000000..ed80bdf28d0
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/Makefile
@@ -0,0 +1,24 @@
+# contrib/ext_vacuum_statistics/Makefile
+
+EXTENSION = ext_vacuum_statistics
+MODULE_big = ext_vacuum_statistics
+OBJS = vacuum_statistics.o
+DATA = ext_vacuum_statistics--1.0.sql
+PGFILEDESC = "ext_vacuum_statistics - convenience views for extended vacuum statistics"
+
+ISOLATION = vacuum-extending-in-repetable-read
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
+TAP_TESTS = 1
+
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/ext_vacuum_statistics
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/ext_vacuum_statistics/README.md b/contrib/ext_vacuum_statistics/README.md
new file mode 100644
index 00000000000..51697eab023
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/README.md
@@ -0,0 +1,165 @@
+# ext_vacuum_statistics
+
+Extended vacuum statistics extension for PostgreSQL. It collects and exposes detailed per-table, per-index, and per-database vacuum statistics (buffer I/O, WAL, general, timing) via convenient views in the `ext_vacuum_statistics` schema.
+
+## Installation
+
+```
+./configure tmp_install="$(pwd)/my/inst"
+make clean && make && make install
+cd contrib/ext_vacuum_statistics
+make && make install
+```
+
+It is essential that the extension is listed in `shared_preload_libraries` because it registers a vacuum hook at server startup.
+
+In your `postgresql.conf`:
+
+```
+shared_preload_libraries = 'ext_vacuum_statistics'
+```
+
+Restart PostgreSQL.
+
+In your database:
+
+```sql
+CREATE EXTENSION ext_vacuum_statistics;
+```
+
+## Usage
+
+Query vacuum statistics via the provided views:
+
+```sql
+-- Per-table heap vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+
+-- Per-index vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+
+-- Per-database aggregate vacuum statistics
+SELECT * FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+```
+
+Example output:
+
+```
+ relname | total_blks_read | total_blks_hit | wal_records | tuples_deleted | pages_removed
+-----------+-----------------+----------------+-------------+----------------+---------------
+ mytable | 120 | 340 | 15 | 500 | 10
+```
+
+Reset statistics when needed:
+
+```sql
+SELECT ext_vacuum_statistics.vacuum_statistics_reset();
+```
+
+## Configuration (GUCs)
+
+| GUC | Default | Description |
+|-----|---------|-------------|
+| `vacuum_statistics.enabled` | on | Enable extended vacuum statistics collection |
+| `vacuum_statistics.object_types` | all | Object types for statistics: `all`, `databases`, `relations` |
+| `vacuum_statistics.track_relations` | all | When tracking relations: `all`, `system`, `user` |
+| `vacuum_statistics.track_databases_from_list` | off | If on, track only databases added via add_track_database |
+| `vacuum_statistics.track_relations_from_list` | off | If on, track only relations added via add_track_relation |
+
+## Memory usage
+
+Each tracked object (table, index, or database) uses approximately **232 bytes** of shared memory on Linux x86_64 (e.g. Ubuntu): common stats (buffers, WAL, timing) ~144 bytes; type + union ~88 bytes (union holds table-specific or index-specific fields, allocated size is the same for both).
+
+The exact size depends on the platform. Call `ext_vacuum_statistics.shared_memory_size()` to get the total shared memory used by the extension. The GUCs provided by the extension allow controlling the amount of memory used: `vacuum_statistics.object_types` to track only databases or relations, `vacuum_statistics.track_relations` to restrict to user or system tables/indexes, and `track_*_from_list` to track only selected databases and relations.
+
+Example: a database with 1000 tables and 2000 indexes, all tracked, uses about **700 KB** on Ubuntu (3001 entries × 232 bytes). Per-database entries add one entry per tracked database.
+
+## Advanced tuning
+
+### Track only database-level stats
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+Statistics are accumulated per database; per-relation views remain empty.
+
+### Track only user or system tables
+
+```sql
+SET vacuum_statistics.object_types = 'relations';
+SET vacuum_statistics.track_relations = 'user'; -- skip system catalogs
+-- or
+SET vacuum_statistics.track_relations = 'system'; -- only system catalogs
+```
+
+### Filter by database or relation OIDs
+
+Add OIDs via functions (persisted to `pg_stat/ext_vacuum_statistics_track.oid`) and enable filtering:
+
+```sql
+-- Add databases and relations to track
+SELECT ext_vacuum_statistics.add_track_database(16384);
+SELECT ext_vacuum_statistics.add_track_relation(16384, 16385); -- dboid, reloid
+SELECT ext_vacuum_statistics.add_track_relation(0, 16386); -- rel 16386 in any db
+
+-- Enable list-based filtering (off = track all)
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+Remove OIDs when no longer needed:
+
+```sql
+SELECT ext_vacuum_statistics.remove_track_database(16384);
+SELECT ext_vacuum_statistics.remove_track_relation(16384, 16385);
+```
+
+Inspect the current tracking configuration:
+
+```sql
+SELECT * FROM ext_vacuum_statistics.track_list();
+```
+
+Returns `track_kind`, `dboid`, `reloid`. When `dboid` or `reloid` is NULL, statistics are collected for all.
+
+## Recipes
+
+**Reduce overhead by tracking only databases:**
+
+```sql
+SET vacuum_statistics.object_types = 'databases';
+```
+
+**Track only a specific table in a specific database:**
+
+```sql
+SELECT ext_vacuum_statistics.add_track_database(
+ (SELECT oid FROM pg_database WHERE datname = current_database())
+);
+SELECT ext_vacuum_statistics.add_track_relation(
+ (SELECT oid FROM pg_database WHERE datname = current_database()),
+ 'mytable'::regclass
+);
+SET vacuum_statistics.track_databases_from_list = on;
+SET vacuum_statistics.track_relations_from_list = on;
+```
+
+**Disable statistics collection temporarily:**
+
+```sql
+SET vacuum_statistics.enabled = off;
+```
+
+## Views
+
+| View | Description |
+|------|-------------|
+| `ext_vacuum_statistics.pg_stats_vacuum_tables` | Per-table heap vacuum stats (pages scanned, tuples deleted, WAL, timing, etc.) |
+| `ext_vacuum_statistics.pg_stats_vacuum_indexes` | Per-index vacuum stats |
+| `ext_vacuum_statistics.pg_stats_vacuum_database` | Per-database aggregate vacuum stats |
+
+## Limitations
+
+- Must be loaded via `shared_preload_libraries`; it cannot be loaded on demand.
+- Tracking configuration (`add_track_*`, `remove_track_*`) is stored in a file and shared across all databases in the cluster.
diff --git a/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
new file mode 100644
index 00000000000..89c9594dea8
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/ext_vacuum_statistics.out
@@ -0,0 +1,52 @@
+-- ext_vacuum_statistics regression test
+
+-- Create extension
+CREATE EXTENSION ext_vacuum_statistics;
+
+-- Verify schema and views exist
+SELECT nspname FROM pg_namespace WHERE nspname = 'ext_vacuum_statistics';
+ nspname
+------------------
+ ext_vacuum_statistics
+(1 row)
+
+-- Views should be queryable (may return empty if no vacuum has run)
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT COUNT(*) >= 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Verify views have expected columns
+SELECT COUNT(*) AS tables_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'tables';
+ tables_cols
+-------------
+ 28
+(1 row)
+
+SELECT COUNT(*) AS indexes_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'indexes';
+ indexes_cols
+--------------
+ 20
+(1 row)
+
+SELECT COUNT(*) AS database_cols FROM information_schema.columns
+WHERE table_schema = 'ext_vacuum_statistics' AND table_name = 'database';
+ database_cols
+---------------
+ 15
+(1 row)
diff --git a/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
new file mode 100644
index 00000000000..6b381f9d232
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/expected/vacuum-extending-in-repetable-read.out
@@ -0,0 +1,52 @@
+unused step name: s2_delete
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+-------+--------------+--------------------+------------------+-----------------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 0| 100| 0| 0| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|recently_dead_tuples|missed_dead_tuples|missed_dead_pages|tuples_frozen
+--------------------------+--------------+--------------------+------------------+-----------------+-------------
+test_vacuum_stat_isolation| 100| 100| 0| 0| 101
+(1 row)
+
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
new file mode 100644
index 00000000000..aa3a9ec9699
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics--1.0.sql
@@ -0,0 +1,272 @@
+/*-------------------------------------------------------------------------
+ *
+ * ext_vacuum_statistics--1.0.sql
+ * Extended vacuum statistics via hook and custom storage
+ *
+ * This extension collects extended vacuum statistics via set_report_vacuum_hook
+ * and stores them in shared memory.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+\echo Use "CREATE EXTENSION ext_vacuum_statistics" to load this file. \quit
+
+CREATE SCHEMA IF NOT EXISTS ext_vacuum_statistics;
+
+COMMENT ON SCHEMA ext_vacuum_statistics IS
+ 'Extended vacuum statistics (heap, index, database)';
+
+-- Reset functions
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_entry(
+ dboid oid,
+ relid oid,
+ type int4
+)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'extvac_reset_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.extvac_reset_db_entry(dboid oid)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_reset_db_entry'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.vacuum_statistics_reset()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'vacuum_statistics_reset'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.shared_memory_size()
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'extvac_shared_memory_size'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.shared_memory_size() IS
+ 'Total shared memory in bytes used by the extension for vacuum statistics.';
+
+-- Add/remove OIDs for tracking
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_database(dboid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_database'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_add_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)
+RETURNS boolean
+AS 'MODULE_PATHNAME', 'evs_remove_track_relation'
+LANGUAGE C STRICT;
+
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.track_list()
+RETURNS TABLE(track_kind text, dboid oid, reloid oid)
+AS 'MODULE_PATHNAME', 'evs_track_list'
+LANGUAGE C STRICT;
+
+COMMENT ON FUNCTION ext_vacuum_statistics.track_list() IS
+ 'List of database and relation OIDs for which vacuum statistics are collected.';
+
+-- Track-list mutation requires superuser or pg_read_all_stats; hide the
+-- functions from PUBLIC so the error is also produced for ordinary users
+-- before the C-level privilege check runs.
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) FROM PUBLIC;
+REVOKE ALL ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_database(oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.add_track_relation(oid, oid) TO pg_read_all_stats;
+GRANT EXECUTE ON FUNCTION ext_vacuum_statistics.remove_track_relation(oid, oid) TO pg_read_all_stats;
+
+-- Internal C function to fetch table vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_tables(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_scanned bigint,
+ OUT pages_removed bigint,
+ OUT vm_new_frozen_pages bigint,
+ OUT vm_new_visible_pages bigint,
+ OUT vm_new_visible_frozen_pages bigint,
+ OUT tuples_frozen bigint,
+ OUT recently_dead_tuples bigint,
+ OUT index_vacuum_count bigint,
+ OUT missed_dead_pages bigint,
+ OUT missed_dead_tuples bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_tables'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch index vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_indexes(
+ IN dboid oid,
+ IN reloid oid,
+ OUT relid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT rel_blks_read bigint,
+ OUT rel_blks_hit bigint,
+ OUT tuples_deleted bigint,
+ OUT pages_deleted bigint
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_indexes'
+LANGUAGE C STRICT STABLE;
+
+-- Internal C function to fetch database vacuum stats
+CREATE OR REPLACE FUNCTION ext_vacuum_statistics.pg_stats_get_vacuum_database(
+ IN dboid oid,
+ OUT dbid oid,
+ OUT total_blks_read bigint,
+ OUT total_blks_hit bigint,
+ OUT total_blks_dirtied bigint,
+ OUT total_blks_written bigint,
+ OUT wal_records bigint,
+ OUT wal_fpi bigint,
+ OUT wal_bytes numeric,
+ OUT blk_read_time double precision,
+ OUT blk_write_time double precision,
+ OUT delay_time double precision,
+ OUT total_time double precision,
+ OUT wraparound_failsafe_count integer,
+ OUT interrupts_count integer
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stats_get_vacuum_database'
+LANGUAGE C STRICT STABLE;
+
+-- View: vacuum statistics per table (heap)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_tables AS
+SELECT
+ rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname AS relname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.vm_new_frozen_pages,
+ stats.vm_new_visible_pages,
+ stats.vm_new_visible_frozen_pages,
+ stats.tuples_frozen,
+ stats.recently_dead_tuples,
+ stats.index_vacuum_count,
+ stats.missed_dead_pages,
+ stats.missed_dead_tuples
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_tables(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'r'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_tables IS
+ 'Extended vacuum statistics per table (heap)';
+
+-- View: vacuum statistics per index
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid AS indexrelid,
+ ns.nspname AS schema,
+ rel.relname AS indexrelname,
+ db.datname AS dbname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.total_time,
+ stats.wraparound_failsafe_count,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.tuples_deleted,
+ stats.pages_deleted
+FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_indexes(db.oid, rel.oid) stats
+WHERE db.datname = current_database()
+ AND rel.relkind = 'i'
+ AND rel.relnamespace = ns.oid
+ AND rel.oid = stats.relid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_indexes IS
+ 'Extended vacuum statistics per index';
+
+-- View: vacuum statistics per database (aggregate)
+CREATE VIEW ext_vacuum_statistics.pg_stats_vacuum_database AS
+SELECT
+ db.oid AS dboid,
+ db.datname AS dbname,
+ stats.total_blks_read AS db_blks_read,
+ stats.total_blks_hit AS db_blks_hit,
+ stats.total_blks_dirtied AS db_blks_dirtied,
+ stats.total_blks_written AS db_blks_written,
+ stats.wal_records AS db_wal_records,
+ stats.wal_fpi AS db_wal_fpi,
+ stats.wal_bytes AS db_wal_bytes,
+ stats.blk_read_time AS db_blk_read_time,
+ stats.blk_write_time AS db_blk_write_time,
+ stats.delay_time AS db_delay_time,
+ stats.total_time AS db_total_time,
+ stats.wraparound_failsafe_count AS db_wraparound_failsafe_count,
+ stats.interrupts_count
+FROM pg_database db
+LEFT JOIN LATERAL ext_vacuum_statistics.pg_stats_get_vacuum_database(db.oid) stats ON db.oid = stats.dbid;
+
+COMMENT ON VIEW ext_vacuum_statistics.pg_stats_vacuum_database IS
+ 'Extended vacuum statistics per database (aggregate)';
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
new file mode 100644
index 00000000000..9b711487623
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.conf
@@ -0,0 +1,2 @@
+# Config for ext_vacuum_statistics regression tests
+shared_preload_libraries = 'ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
new file mode 100644
index 00000000000..518350a64b7
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/ext_vacuum_statistics.control
@@ -0,0 +1,5 @@
+# ext_vacuum_statistics extension
+comment = 'Extended vacuum statistics via hook (requires shared_preload_libraries)'
+default_version = '1.0'
+relocatable = true
+module_pathname = '$libdir/ext_vacuum_statistics'
diff --git a/contrib/ext_vacuum_statistics/meson.build b/contrib/ext_vacuum_statistics/meson.build
new file mode 100644
index 00000000000..72338baa500
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/meson.build
@@ -0,0 +1,41 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+#
+# ext_vacuum_statistics - extended vacuum statistics via hook
+# Requires shared_preload_libraries = 'ext_vacuum_statistics'
+
+ext_vacuum_statistics_sources = files(
+ 'vacuum_statistics.c',
+)
+
+ext_vacuum_statistics = shared_module('ext_vacuum_statistics',
+ ext_vacuum_statistics_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += ext_vacuum_statistics
+
+install_data(
+ 'ext_vacuum_statistics.control',
+ 'ext_vacuum_statistics--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'ext_vacuum_statistics',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'isolation': {
+ 'specs': [
+ 'vacuum-extending-in-repetable-read',
+ ],
+ 'regress_args': ['--temp-config', files('ext_vacuum_statistics.conf')],
+ 'runningcheck': false,
+ },
+ 'tap': {
+ 'tests': [
+ 't/052_vacuum_extending_basic_test.pl',
+ 't/053_vacuum_extending_freeze_test.pl',
+ ],
+ },
+}
diff --git a/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
new file mode 100644
index 00000000000..4891e248cca
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/specs/vacuum-extending-in-repetable-read.spec
@@ -0,0 +1,59 @@
+# Test for checking recently_dead_tuples, tuples_deleted and frozen tuples in ext_vacuum_statistics.pg_stats_vacuum_tables.
+# recently_dead_tuples values are counted when vacuum hasn't cleared tuples because they were deleted recently.
+# recently_dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ CREATE EXTENSION ext_vacuum_statistics;
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP EXTENSION ext_vacuum_statistics CASCADE;
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+setup {
+ SET track_io_timing = on;
+}
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+}
+step s1_commit { COMMIT; }
+
+session s2
+setup {
+ SET track_io_timing = on;
+}
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.recently_dead_tuples, vt.missed_dead_tuples, vt.missed_dead_pages, vt.tuples_frozen
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
diff --git a/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
new file mode 100644
index 00000000000..9463d5145f4
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/052_vacuum_extending_basic_test.pl
@@ -0,0 +1,780 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+# Test cumulative vacuum stats system using TAP
+#
+# This test validates the accuracy and behavior of cumulative vacuum statistics
+# across heap tables, indexes, and databases using:
+#
+# • ext_vacuum_statistics.pg_stats_vacuum_tables
+# • ext_vacuum_statistics.pg_stats_vacuum_indexes
+# • ext_vacuum_statistics.pg_stats_vacuum_database
+#
+# A polling helper function repeatedly checks the stats views until expected
+# deltas appear or a configurable timeout expires. This guarantees that
+# stats-collector propagation delays do not lead to flaky test behavior.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test harness setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and logging level
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+my $stderr;
+my $base_stats;
+my $wals;
+my $ibase_stats;
+my $iwals;
+
+$node->start(
+ '>' => \$base_stats,
+ '2>' => \$stderr
+);
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+# Main test database name and number of rows to insert
+my $dbname = 'statistic_vacuum_database_regression';
+my $size_tab = 1000;
+
+# Enable required session settings and force the stats collector to flush next
+$node->safe_psql($dbname, q{
+ SET track_functions = 'all';
+ SELECT pg_stat_force_next_flush();
+});
+
+#------------------------------------------------------------------------------
+# Create test table and populate it
+#------------------------------------------------------------------------------
+
+$node->safe_psql(
+ $dbname,
+ "CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE vestat (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ ANALYZE vestat;"
+);
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables and ext_vacuum_statistics.pg_stats_vacuum_indexes until both the
+# table-level and index-level counters exceed the provided baselines, or until
+# the configured timeout elapses.
+#
+# Expected named args (baseline values):
+# tab_tuples_deleted
+# tab_wal_records
+# idx_tuples_deleted
+# idx_wal_records
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+ my $tab_tuples_deleted = ($args{tab_tuples_deleted} or 0);
+ my $tab_wal_records = ($args{tab_wal_records} or 0);
+ my $idx_tuples_deleted = ($args{idx_tuples_deleted} or 0);
+ my $idx_wal_records = ($args{idx_wal_records} or 0);
+
+ my $start = time();
+ while ((time() - $start) < $timeout) {
+
+ my $result_query = $node->safe_psql(
+ $dbname,
+ "VACUUM vestat;
+ SELECT
+ (SELECT (tuples_deleted > $tab_tuples_deleted AND wal_records > $tab_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat')
+ AND
+ (SELECT (tuples_deleted > $idx_tuples_deleted AND wal_records > $idx_wal_records)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey');"
+ );
+
+ return 1 if ($result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum-stat snapshots for later comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+my $tuples_deleted = 0;
+my $pages_scanned = 0;
+my $pages_removed = 0;
+my $wal_records = 0;
+my $wal_bytes = 0;
+my $wal_fpi = 0;
+
+my $index_tuples_deleted = 0;
+my $index_pages_deleted = 0;
+my $index_wal_records = 0;
+my $index_wal_bytes = 0;
+my $index_wal_fpi = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+my $tuples_deleted_prev = 0;
+my $pages_scanned_prev = 0;
+my $pages_removed_prev = 0;
+my $wal_records_prev = 0;
+my $wal_bytes_prev = 0;
+my $wal_fpi_prev = 0;
+
+my $index_tuples_deleted_prev = 0;
+my $index_pages_deleted_prev = 0;
+my $index_wal_records_prev = 0;
+my $index_wal_bytes_prev = 0;
+my $index_wal_fpi_prev = 0;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Reads current values of relevant vacuum counters for the test table and its
+# primary index, storing them in package variables for subsequent comparisons.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ # fetch actual base vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT vm_new_visible_frozen_pages, tuples_deleted, pages_scanned, pages_removed, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+ );
+
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($vm_new_visible_frozen_pages, $tuples_deleted, $pages_scanned, $pages_removed, $wal_records, $wal_bytes, $wal_fpi)
+ = split /\s+/, $base_statistics;
+
+ # --- index stats ---
+ my $index_base_statistics = $node->safe_psql(
+ $dbname,
+ "SELECT tuples_deleted, pages_deleted, wal_records, wal_bytes, wal_fpi
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+ );
+
+ $index_base_statistics =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($index_tuples_deleted, $index_pages_deleted, $index_wal_records, $index_wal_bytes, $index_wal_fpi)
+ = split /\s+/, $index_base_statistics;
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#
+# Save current values (previously fetched by fetch_vacuum_stats) so that we
+# later fetch new values and compare them.
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $tuples_deleted_prev = $tuples_deleted;
+ $pages_scanned_prev = $pages_scanned;
+ $pages_removed_prev = $pages_removed;
+ $wal_records_prev = $wal_records;
+ $wal_bytes_prev = $wal_bytes;
+ $wal_fpi_prev = $wal_fpi;
+
+ $index_tuples_deleted_prev = $index_tuples_deleted;
+ $index_pages_deleted_prev = $index_pages_deleted;
+ $index_wal_records_prev = $index_wal_records;
+ $index_wal_bytes_prev = $index_wal_bytes;
+ $index_wal_fpi_prev = $index_wal_fpi;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#
+# Print values in case of an error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " tuples_deleted = $tuples_deleted_prev\n" .
+ " pages_scanned = $pages_scanned_prev\n" .
+ " pages_removed = $pages_removed_prev\n" .
+ " wal_records = $wal_records_prev\n" .
+ " wal_bytes = $wal_bytes_prev\n" .
+ " wal_fpi = $wal_fpi_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " tuples_deleted = $tuples_deleted\n" .
+ " pages_scanned = $pages_scanned\n" .
+ " pages_removed = $pages_removed\n" .
+ " wal_records = $wal_records\n" .
+ " wal_bytes = $wal_bytes\n" .
+ " wal_fpi = $wal_fpi\n" .
+ "Index statistics:\n" .
+ " Before test:\n" .
+ " tuples_deleted = $index_tuples_deleted_prev\n" .
+ " pages_deleted = $index_pages_deleted_prev\n" .
+ " wal_records = $index_wal_records_prev\n" .
+ " wal_bytes = $index_wal_bytes_prev\n" .
+ " wal_fpi = $index_wal_fpi_prev\n" .
+ " After test:\n" .
+ " tuples_deleted = $index_tuples_deleted\n" .
+ " pages_deleted = $index_pages_deleted\n" .
+ " wal_records = $index_wal_records\n" .
+ " wal_bytes = $index_wal_bytes\n" .
+ " wal_fpi = $index_wal_fpi\n"
+ );
+};
+
+sub fetch_error_base_db_vacuum_statistics {
+ my (%args) = @_;
+
+ # Validate presence of required args (allow 0 as valid numeric baseline)
+ die "database name required"
+ unless exists $args{database_name} && defined $args{database_name};
+ my $database_name = $args{database_name};
+
+ # fetch actual base database vacuum statistics
+ my $base_statistics = $node->safe_psql(
+ $database_name,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+ );
+ $base_statistics =~ s/\s*\|\s*/ /g; # transform " | " in space
+ my ($db_blks_hit, $total_blks_dirtied, $total_blks_written,
+ $wal_records, $wal_fpi, $wal_bytes) = split /\s+/, $base_statistics;
+
+ diag(
+ "BASE STATS MISMATCH FOR DATABASE $dbname:\n" .
+ " db_blks_hit = $db_blks_hit\n" .
+ " total_blks_dirtied = $total_blks_dirtied\n" .
+ " total_blks_written = $total_blks_written\n" .
+ " wal_records = $wal_records\n" .
+ " wal_fpi = $wal_fpi\n" .
+ " wal_bytes = $wal_bytes\n"
+ );
+}
+
+
+#------------------------------------------------------------------------------
+# Test 1: Delete half the rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 1: Delete half the rows, run VACUUM' => sub
+{
+
+$node->safe_psql($dbname, "DELETE FROM vestat WHERE x % 2 = 0;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+# Poll the stats view until expected deltas appear or timeout
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => 0,
+ tab_wal_records => 0,
+ idx_tuples_deleted => 0,
+ idx_wal_records => 0,
+);
+ok($updated, 'vacuum stats updated after vacuuming half-deleted table (tuples_deleted and wal_fpi advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming half-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Delete all rows, run VACUUM, and wait for stats to advance
+#------------------------------------------------------------------------------
+subtest 'Test 2: Delete all rows, run VACUUM' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, "DELETE FROM vestat;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after vacuuming all-deleted table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds after vacuuming all-deleted table";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed > $pages_removed_prev, 'table pages_removed has increased');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > 0, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Test VACUUM FULL — it should not report to the stats collector
+#------------------------------------------------------------------------------
+subtest 'Test 3: Test VACUUM FULL — it should not report to the stats collector' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ DELETE FROM vestat;
+ VACUUM FULL vestat;"
+);
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records == $wal_records_prev, 'table wal_records stay the same');
+ok($wal_bytes == $wal_bytes_prev, 'table wal_bytes stay the same');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting
+#------------------------------------------------------------------------------
+subtest 'Test 4: Update table, checkpoint, and VACUUM to provoke WAL/FPI accounting' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ CHECKPOINT;
+ UPDATE vestat SET x = x + 1000;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_tuples_deleted => $tuples_deleted_prev,
+ tab_wal_records => $wal_records_prev,
+ idx_tuples_deleted => $index_tuples_deleted_prev,
+ idx_wal_records => $index_wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples in the table (tuples_deleted and wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted > $tuples_deleted_prev, 'table tuples_deleted has increased');
+ok($pages_scanned > $pages_scanned_prev, 'table pages_scanned has increased');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi > $wal_fpi_prev, 'table wal_fpi has increased');
+
+ok($index_pages_deleted > $index_pages_deleted_prev, 'index pages_deleted has increased');
+ok($index_tuples_deleted > $index_tuples_deleted_prev, 'index tuples_deleted has increased');
+ok($index_wal_records > $index_wal_records_prev, 'index wal_records has increased');
+ok($index_wal_bytes > $index_wal_bytes_prev, 'index wal_bytes has increased');
+ok($index_wal_fpi > $index_wal_fpi_prev, 'index wal_fpi has increased');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 5: Update table, trancate and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 5: Update table, trancate and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ UPDATE vestat SET x = x + 1000;"
+);
+$node->safe_psql($dbname, "TRUNCATE vestat;");
+$node->safe_psql($dbname, "CHECKPOINT;");
+$node->safe_psql($dbname, "VACUUM vestat;");
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records_prev,
+);
+
+ok($updated, 'vacuum stats updated after updating tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 6: Delete all tuples from table, trancate, and vacuuming
+#------------------------------------------------------------------------------
+subtest 'Test 6: Delete all tuples from table, trancate, and vacuuming' => sub
+{
+
+save_vacuum_stats();
+
+$node->safe_psql(
+ $dbname,
+ "INSERT INTO vestat SELECT x FROM generate_series(1, $size_tab) AS g(x);
+ DELETE FROM vestat;
+ TRUNCATE vestat;
+ CHECKPOINT;
+ VACUUM vestat;"
+);
+
+$updated = wait_for_vacuum_stats(
+ tab_wal_records => $wal_records,
+);
+
+ok($updated, 'vacuum stats updated after deleting all tuples and trancation in the table (wal_records advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($tuples_deleted == $tuples_deleted_prev, 'table tuples_deleted stay the same');
+ok($pages_scanned == $pages_scanned_prev, 'table pages_scanned stay the same');
+ok($pages_removed == $pages_removed_prev, 'table pages_removed stay the same');
+ok($wal_records > $wal_records_prev, 'table wal_records has increased');
+ok($wal_bytes > $wal_bytes_prev, 'table wal_bytes has increased');
+ok($wal_fpi == $wal_fpi_prev, 'table wal_fpi stay the same');
+
+ok($index_pages_deleted == $index_pages_deleted_prev, 'index pages_deleted stay the same');
+ok($index_tuples_deleted == $index_tuples_deleted_prev, 'index tuples_deleted stay the same');
+ok($index_wal_records == $index_wal_records_prev, 'index wal_records stay the same');
+ok($index_wal_bytes == $index_wal_bytes_prev, 'index wal_bytes stay the same');
+ok($index_wal_fpi == $index_wal_fpi_prev, 'index wal_fpi stay the same');
+
+} or print_vacuum_stats_on_error();
+
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+#-------------------------------------------------------------------------------------------------------
+# Test 7: Check if we return single vacuum statistics for particular relation from the current database
+#-------------------------------------------------------------------------------------------------------
+subtest 'Test 7: Check if we return vacuum statistics from the current database' => sub
+{
+save_vacuum_stats();
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat';
+ }
+);
+
+# Check if we can get vacuum statistics of particular heap relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'heap vacuum stats return from the current relation and database as expected');
+
+$reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'vestat_pkey';
+ }
+);
+
+# Check if we can get vacuum statistics of particular index relation in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);"
+);
+is($base_stats, 1, 'index vacuum stats return from the current relation and database as expected');
+
+# Check if we return empty results if vacuum statistics with particular oid doesn't exist
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'table vacuum stats return no rows, as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 1);"
+);
+is($base_stats, 0, 'index vacuum stats return no rows, as expected');
+
+# Check if we can get vacuum statistics of all relations in the current database
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_tables;"
+);
+ok($base_stats eq 't', 'vacuum stats per all heap objects available');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT count(*) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_indexes;"
+);
+ok($base_stats eq 't', 'vacuum stats per all index objects available');
+};
+
+#------------------------------------------------------------------------------
+# Test 8: Check relation-level vacuum statistics from another database
+#------------------------------------------------------------------------------
+subtest 'Test 8: Check relation-level vacuum statistics from another database' => sub
+{
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes
+ WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'check the printing index vacuum extended statistics from another database are not available');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'check the printing heap vacuum extended statistics from another database are not available');
+
+# Check that relations from another database are not visible in the view when querying from postgres
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'vestat';"
+);
+is($base_stats, 0, 'vacuum stats per all tables objects from another database are not available as expected');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelname = 'vestat_pkey';"
+);
+is($base_stats, 0, 'vacuum stats per all index objects from another database are not available as expected');
+};
+
+#--------------------------------------------------------------------------------------
+# Test 9: Check database-level vacuum statistics from the current and another database
+#--------------------------------------------------------------------------------------
+subtest 'Test 9: Check database-level vacuum statistics from the current and another database' => sub
+{
+my $db_blk_hit = 0;
+my $total_blks_dirtied = 0;
+my $total_blks_written = 0;
+my $wal_records = 0;
+my $wal_fpi = 0;
+my $wal_bytes = 0;
+$base_stats = $node->safe_psql(
+ $dbname,
+ "SELECT db_blks_hit, db_blks_dirtied,
+ db_blks_written, db_wal_records,
+ db_wal_fpi, db_wal_bytes
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+$base_stats =~ s/\s*\|\s*/ /g; # transform " | " into space
+ ($db_blk_hit, $total_blks_dirtied, $total_blks_written, $wal_records, $wal_fpi, $wal_bytes)
+ = split /\s+/, $base_stats;
+
+ok($db_blk_hit > 0, 'db_blks_hit is more than 0');
+ok($total_blks_dirtied > 0, 'total_blks_dirtied is more than 0');
+ok($total_blks_written > 0, 'total_blks_written is more than 0');
+ok($wal_records > 0, 'wal_records is more than 0');
+ok($wal_fpi > 0, 'wal_fpi is more than 0');
+ok($wal_bytes > 0, 'wal_bytes is more than 0');
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ "SELECT count(*) = 1
+ FROM ext_vacuum_statistics.pg_stats_vacuum_database, pg_database
+ WHERE pg_database.datname = '$dbname'
+ AND pg_database.oid = ext_vacuum_statistics.pg_stats_vacuum_database.dboid;"
+);
+ok($base_stats eq 't', 'check database-level vacuum stats from another database are available');
+};
+
+#------------------------------------------------------------------------------
+# Test 10: Cleanup checks: ensure functions return empty sets for OID = 0
+#------------------------------------------------------------------------------
+subtest 'Test 10: Cleanup checks: ensure functions return empty sets for OID = 0' => sub
+{
+my $dboid = $node->safe_psql(
+ $dbname,
+ "SELECT oid FROM pg_database WHERE datname = current_database();"
+);
+
+# Vacuum statistics for invalid relation OID return empty
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per heap from invalid relation OID return empty as expected');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), 0);
+ }
+);
+is($base_stats, 0, 'vacuum stats per index from invalid relation OID return empty as expected');
+
+$node->safe_psql($dbname, q{
+ DROP TABLE vestat CASCADE;
+ VACUUM;
+});
+
+# Check that we don't print vacuum statistics for deleted objects
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_tables correctly returns no rows for OID = 0');
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT COUNT(*)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_indexes WHERE indexrelid = 0;
+ }
+);
+is($base_stats, 0, 'ext_vacuum_statistics.pg_stats_vacuum_indexes correctly returns no rows for OID = 0');
+
+my $reloid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend';
+ }
+);
+
+$node->safe_psql($dbname, "VACUUM pg_shdepend;");
+
+# Check if we can get vacuum statistics for cluster relations (shared catalogs)
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_tables((SELECT oid FROM pg_database WHERE datname = current_database()), $reloid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common heap objects available');
+
+my $indoid = $node->safe_psql(
+ $dbname,
+ q{
+ SELECT oid FROM pg_class WHERE relname = 'pg_shdepend_reference_index';
+ }
+);
+
+$base_stats = $node->safe_psql(
+ $dbname,
+ qq{
+ SELECT count(*) > 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_indexes((SELECT oid FROM pg_database WHERE datname = current_database()), $indoid);
+ }
+);
+
+is($base_stats, 't', 'vacuum stats for common index objects available');
+
+$node->safe_psql('postgres',
+ "DROP DATABASE $dbname;
+ VACUUM;"
+);
+
+$base_stats = $node->safe_psql(
+ 'postgres',
+ q{
+ SELECT count(*) = 0
+ FROM ext_vacuum_statistics.pg_stats_get_vacuum_database(0);
+ }
+);
+is($base_stats, 't', 'vacuum stats from database with invalid database OID return empty, as expected');
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
new file mode 100644
index 00000000000..4f8f025c63e
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/053_vacuum_extending_freeze_test.pl
@@ -0,0 +1,285 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test cumulative vacuum stats using ext_vacuum_statistics extension (TAP)
+#
+# In short, this test validates the correctness and stability of cumulative
+# vacuum statistics accounting around freezing, visibility, and revision
+# tracking across multiple VACUUMs and backend operations.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum');
+$node->init;
+
+# Configure the server: preload extension and aggressive freezing behavior
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+ vacuum_freeze_min_age = 0
+ vacuum_freeze_table_age = 0
+ vacuum_multixact_freeze_min_age = 0
+ vacuum_multixact_freeze_table_age = 0
+ vacuum_max_eager_freeze_failure_rate = 1.0
+ vacuum_failsafe_age = 0
+ vacuum_multixact_failsafe_age = 0
+ track_functions = 'all'
+});
+
+$node->start();
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_database_regression;
+});
+
+# Main test database name
+my $dbname = 'statistic_vacuum_database_regression';
+
+# Create extension
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+});
+
+#------------------------------------------------------------------------------
+# Timing parameters for polling loops
+#------------------------------------------------------------------------------
+
+my $timeout = 30; # overall wait timeout in seconds
+my $interval = 0.015; # poll interval in seconds (15 ms)
+my $start_time = time();
+my $updated = 0;
+
+#------------------------------------------------------------------------------
+# wait_for_vacuum_stats
+#
+# Polls ext_vacuum_statistics.pg_stats_vacuum_tables until the named columns exceed the
+# provided baseline values or until timeout.
+#
+# tab_all_frozen_pages_count => 0 # baseline numeric
+# tab_all_visible_pages_count => 0 # baseline numeric
+# run_vacuum => 0 # if true, run vacuum before polling
+#
+# Returns: 1 if the condition is met before timeout, 0 otherwise.
+#------------------------------------------------------------------------------
+sub wait_for_vacuum_stats {
+ my (%args) = @_;
+
+ my $tab_all_frozen_pages_count = $args{tab_all_frozen_pages_count} || 0;
+ my $tab_all_visible_pages_count = $args{tab_all_visible_pages_count} || 0;
+ my $run_vacuum = $args{run_vacuum} ? 1 : 0;
+ my $result_query;
+
+ my $start = time();
+ my $sql;
+
+ # Run VACUUM once if requested, before polling
+ if ($run_vacuum) {
+ $node->safe_psql($dbname, 'VACUUM (FREEZE, VERBOSE) vestat');
+ }
+
+ while ((time() - $start) < $timeout) {
+
+ if ($run_vacuum) {
+ $sql = "
+ SELECT (vm_new_visible_frozen_pages > $tab_all_frozen_pages_count)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'vestat'";
+ }
+ else {
+ $sql = "
+ SELECT (pg_stat_get_frozen_page_marks_cleared(c.oid) > $tab_all_frozen_pages_count AND
+ pg_stat_get_visible_page_marks_cleared(c.oid) > $tab_all_visible_pages_count)
+ FROM pg_class c
+ WHERE relname = 'vestat'";
+ }
+
+ $result_query = $node->safe_psql($dbname, $sql);
+
+ return 1 if (defined $result_query && $result_query eq 't');
+
+ sleep($interval);
+ }
+
+ return 0;
+}
+
+#------------------------------------------------------------------------------
+# Variables to hold vacuum statistics snapshots for comparisons
+#------------------------------------------------------------------------------
+
+my $vm_new_visible_frozen_pages = 0;
+
+my $rev_all_frozen_pages = 0;
+my $rev_all_visible_pages = 0;
+
+my $vm_new_visible_frozen_pages_prev = 0;
+
+my $rev_all_frozen_pages_prev = 0;
+my $rev_all_visible_pages_prev = 0;
+
+my $res;
+
+#------------------------------------------------------------------------------
+# fetch_vacuum_stats
+#
+# Loads current values of the relevant vacuum counters for the test table
+# into the package-level variables above so tests can compare later.
+#------------------------------------------------------------------------------
+
+sub fetch_vacuum_stats {
+ $vm_new_visible_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT vt.vm_new_visible_frozen_pages
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables vt
+ WHERE vt.relname = 'vestat';"
+ );
+
+ $rev_all_frozen_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_frozen_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+
+ $rev_all_visible_pages = $node->safe_psql(
+ $dbname,
+ "SELECT pg_stat_get_visible_page_marks_cleared(c.oid)
+ FROM pg_class c
+ WHERE c.relname = 'vestat';"
+ );
+}
+
+#------------------------------------------------------------------------------
+# save_vacuum_stats
+#------------------------------------------------------------------------------
+sub save_vacuum_stats {
+ $vm_new_visible_frozen_pages_prev = $vm_new_visible_frozen_pages;
+ $rev_all_frozen_pages_prev = $rev_all_frozen_pages;
+ $rev_all_visible_pages_prev = $rev_all_visible_pages;
+}
+
+#------------------------------------------------------------------------------
+# print_vacuum_stats_on_error
+#------------------------------------------------------------------------------
+sub print_vacuum_stats_on_error {
+ diag(
+ "Statistics in the failed test\n" .
+ "Table statistics:\n" .
+ " Before test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages_prev\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages_prev\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages_prev\n" .
+ " After test:\n" .
+ " vm_new_visible_frozen_pages = $vm_new_visible_frozen_pages\n" .
+ " rev_all_frozen_pages = $rev_all_frozen_pages\n" .
+ " rev_all_visible_pages = $rev_all_visible_pages\n"
+ );
+};
+
+#------------------------------------------------------------------------------
+# Test 1: Create test table, populate it and run an initial vacuum to force freezing
+#------------------------------------------------------------------------------
+
+subtest 'Test 1: Create test table, populate it and run an initial vacuum to force freezing' => sub
+{
+$node->safe_psql($dbname, q{
+ CREATE TABLE vestat (x int)
+ WITH (autovacuum_enabled = off, fillfactor = 10);
+ INSERT INTO vestat SELECT x FROM generate_series(1, 1000) AS g(x);
+ ANALYZE vestat;
+ VACUUM (FREEZE, VERBOSE) vestat;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 2: Trigger backend updates
+# Backend activity should reset per-page visibility/freeze marks and increment revision counters
+#------------------------------------------------------------------------------
+subtest 'Test 2: Trigger backend updates' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{
+ UPDATE vestat SET x = x + 1001;
+});
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => 0,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 0,
+);
+
+ok($updated,
+ 'vacuum stats updated after backend tuple updates (rev_all_frozen_pages and rev_all_visible_pages advanced)')
+ or diag "Timeout waiting for vacuum stats update after $timeout seconds";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages == $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages stay the same');
+ok($rev_all_frozen_pages > $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages has increased');
+ok($rev_all_visible_pages > $rev_all_visible_pages_prev, 'table rev_all_visible_pages has increased');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility
+#------------------------------------------------------------------------------
+subtest 'Test 3: Force another vacuum after backend modifications - vacuum should restore freeze/visibility' => sub
+{
+save_vacuum_stats();
+
+$node->safe_psql($dbname, q{ VACUUM vestat; });
+
+$updated = wait_for_vacuum_stats(
+ tab_all_frozen_pages_count => $vm_new_visible_frozen_pages,
+ tab_all_visible_pages_count => 0,
+ run_vacuum => 1,
+);
+
+ok($updated,
+ 'vacuum stats updated after vacuuming the all-updated table (vm_new_visible_frozen_pages advanced)')
+ or diag "Timeout waiting for ext_vacuum_statistics to update after $timeout seconds during vacuum";
+
+fetch_vacuum_stats();
+
+ok($vm_new_visible_frozen_pages > $vm_new_visible_frozen_pages_prev, 'table vm_new_visible_frozen_pages has increased');
+ok($rev_all_frozen_pages == $rev_all_frozen_pages_prev, 'table rev_all_frozen_pages stay the same');
+ok($rev_all_visible_pages == $rev_all_visible_pages_prev, 'table rev_all_visible_pages stay the same');
+} or print_vacuum_stats_on_error();
+
+#------------------------------------------------------------------------------
+# Cleanup
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ DROP DATABASE statistic_vacuum_database_regression;
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
new file mode 100644
index 00000000000..a195249842b
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/t/054_vacuum_extending_gucs_test.pl
@@ -0,0 +1,279 @@
+# Copyright (c) 2025 PostgreSQL Global Development Group
+#
+# Test GUC parameters for ext_vacuum_statistics extension:
+# vacuum_statistics.enabled
+# vacuum_statistics.object_types (all, databases, relations)
+# vacuum_statistics.track_relations (all, system, user)
+# vacuum_statistics.track_databases_from_list, add/remove_track_database
+# add/remove_track_database, add/remove_track_relation, track_*_from_list
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+#------------------------------------------------------------------------------
+# Test cluster setup
+#------------------------------------------------------------------------------
+
+my $node = PostgreSQL::Test::Cluster->new('ext_stat_vacuum_gucs');
+$node->init;
+
+$node->append_conf('postgresql.conf', q{
+ shared_preload_libraries = 'ext_vacuum_statistics'
+ log_min_messages = notice
+});
+
+$node->start;
+
+#------------------------------------------------------------------------------
+# Database creation and initialization
+#------------------------------------------------------------------------------
+
+$node->safe_psql('postgres', q{
+ CREATE DATABASE statistic_vacuum_gucs;
+});
+
+my $dbname = 'statistic_vacuum_gucs';
+
+$node->safe_psql($dbname, q{
+ CREATE EXTENSION ext_vacuum_statistics;
+ CREATE TABLE guc_test (x int PRIMARY KEY)
+ WITH (autovacuum_enabled = off);
+ INSERT INTO guc_test SELECT x FROM generate_series(1, 100) AS g(x);
+ ANALYZE guc_test;
+});
+
+# Get OIDs for filtering tests
+my $dboid = $node->safe_psql($dbname, q{SELECT oid FROM pg_database WHERE datname = current_database()});
+my $reloid = $node->safe_psql($dbname, q{SELECT oid FROM pg_class WHERE relname = 'guc_test'});
+
+#------------------------------------------------------------------------------
+# Reset stats and run vacuum (all in one session so GUCs persist)
+#------------------------------------------------------------------------------
+
+sub reset_and_vacuum {
+ my ($db, $table, $opts) = @_;
+ $table ||= 'guc_test';
+ my $gucs = $opts && $opts->{gucs} ? $opts->{gucs} : [];
+ my $modify = $opts && $opts->{modify};
+ my $extra = $opts && $opts->{extra_vacuum} ? $opts->{extra_vacuum} : [];
+ $extra = [$extra] unless ref $extra eq 'ARRAY';
+ my $sql = join("\n", (map { "SET $_;" } @$gucs),
+ "SELECT ext_vacuum_statistics.vacuum_statistics_reset();",
+ $modify ? (
+ "TRUNCATE $table;",
+ "INSERT INTO $table SELECT x FROM generate_series(1, 100) AS g(x);",
+ "DELETE FROM $table;",
+ ) : (),
+ "VACUUM $table;",
+ (map { "VACUUM $_;" } @$extra),
+ # Make pending stats visible to subsequent sessions without sleeping.
+ "SELECT pg_stat_force_next_flush();");
+ $node->safe_psql($db, $sql);
+}
+
+#------------------------------------------------------------------------------
+# Test 1: vacuum_statistics.enabled
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.enabled' => sub {
+ reset_and_vacuum($dbname);
+
+ # Default: enabled - should have stats
+ my $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($count > 0, 'stats collected when enabled');
+
+ # Disable, reset and vacuum in same session. Assert not only that the
+ # row count is zero, but that the specific counters remain zero: a stray
+ # row with zero counters would otherwise pass a bare COUNT(*)=0 check.
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ['vacuum_statistics.enabled = off'] });
+
+ $count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($count, 0, 'no rows when disabled');
+
+ my $sums = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_dirtied), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($sums, '0', 'no counters accumulated when disabled');
+};
+
+#------------------------------------------------------------------------------
+# Test 2: vacuum_statistics.object_types (databases only, relations only)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.object_types' => sub {
+ # track only db stats, no relation stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'databases'"],
+ modify => 1,
+ });
+ my $db_has_dbs = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_dbs = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_dbs, 0, 'track=databases: no relation stats');
+ ok($db_has_dbs > 0, 'track=databases: database stats collected');
+
+ # track only relation stats, no db stats
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.object_types = 'relations'"],
+ modify => 1,
+ });
+ my $db_has_rels = $node->safe_psql($dbname,
+ "SELECT COALESCE(SUM(db_blks_hit), 0) > 0 FROM ext_vacuum_statistics.pg_stats_vacuum_database WHERE dboid = $dboid");
+ my $rel_rels = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_rels > 0, 'track=relations: relation stats collected');
+ is($db_has_rels, 'f', 'track=relations: no database stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 3: vacuum_statistics.track_relations (system, user)
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.track_relations' => sub {
+ # track_relations - only user tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'user'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ my $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ my $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ ok($user_rel > 0, 'track_relations=user: user table stats collected');
+ is($sys_rel, 0, 'track_relations=user: system table stats not collected');
+
+ # track_relations - only system tables
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => [
+ "vacuum_statistics.object_types = 'relations'",
+ "vacuum_statistics.track_relations = 'system'",
+ ],
+ extra_vacuum => ['pg_class'],
+ });
+
+ $user_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ $sys_rel = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'pg_class'");
+ is($user_rel, 0, 'track_relations=system: user table stats not collected');
+ ok($sys_rel > 0, 'track_relations=system: system table stats collected');
+};
+
+#------------------------------------------------------------------------------
+# Test 4: track_databases (via add/remove_track_database)
+#------------------------------------------------------------------------------
+subtest 'track_databases (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'db in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_database($dboid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_databases_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'db removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 5: track_relations (via add/remove_track_relation)
+#------------------------------------------------------------------------------
+subtest 'track_relations (add/remove)' => sub {
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.add_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ my $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ ok($rel_count > 0, 'table in list: stats collected');
+
+ $node->safe_psql($dbname, "SELECT ext_vacuum_statistics.remove_track_relation($dboid, $reloid)");
+ reset_and_vacuum($dbname, 'guc_test', { gucs => ["vacuum_statistics.track_relations_from_list = on"], modify => 1 });
+
+ $rel_count = $node->safe_psql($dbname,
+ "SELECT COUNT(*) FROM ext_vacuum_statistics.pg_stats_vacuum_tables WHERE relname = 'guc_test'");
+ is($rel_count, 0, 'table removed from list: no stats');
+};
+
+#------------------------------------------------------------------------------
+# Test 6: vacuum_statistics.collect - per-category gating
+#
+# With collect='wal' only wal_* counters must advance; buffer, timing, and
+# general categories must stay at zero. With collect='buffers' the inverse
+# holds. Unknown tokens must be rejected by the check-hook.
+#------------------------------------------------------------------------------
+subtest 'vacuum_statistics.collect' => sub {
+ # wal-only: WAL counters should accumulate, buffers/timing/general should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'wal'"],
+ modify => 1,
+ });
+
+ my $wal = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal, 't', "collect='wal': wal_records accumulated");
+
+ my $other = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0)
+ + COALESCE(SUM(total_time), 0)
+ + COALESCE(SUM(tuples_deleted), 0)
+ + COALESCE(SUM(pages_scanned), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($other, '0',
+ "collect='wal': buffer/timing/general counters not accumulated");
+
+ # buffers-only: buffer counters should advance, WAL should not.
+ reset_and_vacuum($dbname, 'guc_test', {
+ gucs => ["vacuum_statistics.collect = 'buffers'"],
+ modify => 1,
+ });
+
+ my $buf = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(total_blks_read), 0)
+ + COALESCE(SUM(total_blks_hit), 0) > 0
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($buf, 't', "collect='buffers': buffer counters accumulated");
+
+ my $wal_off = $node->safe_psql($dbname, q{
+ SELECT COALESCE(SUM(wal_records), 0)
+ FROM ext_vacuum_statistics.pg_stats_vacuum_tables
+ WHERE relname = 'guc_test'
+ });
+ is($wal_off, '0',
+ "collect='buffers': WAL counters not accumulated");
+
+ # Unknown category must be rejected by the check-hook.
+ my ($ret, $stdout, $stderr) = $node->psql($dbname,
+ "SET vacuum_statistics.collect = 'nope'");
+ isnt($ret, 0, "collect='nope': rejected by check-hook");
+ like($stderr, qr/Unrecognized category "nope"/,
+ "collect='nope': errdetail names the offending token");
+};
+
+$node->stop;
+
+done_testing();
diff --git a/contrib/ext_vacuum_statistics/vacuum_statistics.c b/contrib/ext_vacuum_statistics/vacuum_statistics.c
new file mode 100644
index 00000000000..144b9bcb814
--- /dev/null
+++ b/contrib/ext_vacuum_statistics/vacuum_statistics.c
@@ -0,0 +1,1387 @@
+/*
+ * ext_vacuum_statistics - Extended vacuum statistics for PostgreSQL
+ *
+ * This module collects detailed vacuum statistics (I/O, WAL, timing, etc.)
+ * at relation and database level by hooking into the vacuum reporting path.
+ * Statistics are stored via pgstat custom statistics. Management of statistics
+ * storage and output functions are implemented in this module.
+ */
+#include "postgres.h"
+
+#include "access/transam.h"
+#include "catalog/catalog.h"
+#include "catalog/objectaccess.h"
+#include "catalog/pg_authid.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_database.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+#include "utils/guc.h"
+#include "utils/hsearch.h"
+#include "utils/lsyscache.h"
+#include "utils/pgstat_kind.h"
+#include "utils/pgstat_internal.h"
+#include "utils/tuplestore.h"
+
+#ifdef PG_MODULE_MAGIC
+PG_MODULE_MAGIC;
+#endif
+
+/* Two kinds: relations (tables/indexes) and database aggregates */
+#define PGSTAT_KIND_EXTVAC_RELATION 24
+#define PGSTAT_KIND_EXTVAC_DB 25
+
+#define SJ_NODENAME "vacuum_statistics"
+#define EVS_TRACK_FILENAME "pg_stat/ext_vacuum_statistics_track.oid"
+
+/* Bit flags for evs_track (object_types): 'all', 'databases', 'relations' */
+#define EVS_TRACK_RELATIONS 0x01
+#define EVS_TRACK_DATABASES 0x02
+
+/* Bit flags for evs_track_relations: 'all', 'system', 'user' */
+#define EVS_FILTER_SYSTEM 0x01
+#define EVS_FILTER_USER 0x02
+
+/*
+ * Bit flags for evs_collect_mask. Each category groups counters that can be
+ * accumulated (or skipped) together, letting users reduce overhead at run
+ * time by turning off categories they don't need.
+ */
+#define EVS_COLLECT_BUFFERS 0x1 /* blks_*, blk_*_time */
+#define EVS_COLLECT_WAL 0x2 /* wal_records, wal_fpi, wal_bytes */
+#define EVS_COLLECT_GENERAL 0x4 /* tuples_deleted, pages_*, vm_*,
+ * wraparound_failsafe_count,
+ * interrupts_count */
+#define EVS_COLLECT_TIMING 0x8 /* delay_time, total_time */
+#define EVS_COLLECT_ALL (EVS_COLLECT_BUFFERS | EVS_COLLECT_WAL | \
+ EVS_COLLECT_GENERAL | EVS_COLLECT_TIMING)
+
+/* GUCs */
+static bool evs_enabled = true;
+static char *evs_track = "all"; /* 'all', 'databases', 'relations' */
+static char *evs_track_relations = "all"; /* 'all', 'system', 'user' */
+static int evs_track_bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+static int evs_track_relations_bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+static bool evs_track_databases_from_list = false; /* if true, track only
+ * databases in list */
+static bool evs_track_relations_from_list = false; /* if true, track only
+ * relations in list */
+static char *evs_collect = "all"; /* categories to collect */
+static int evs_collect_mask = EVS_COLLECT_ALL;
+
+/* Hook */
+static set_report_vacuum_hook_type prev_report_vacuum_hook = NULL;
+static object_access_hook_type prev_object_access_hook = NULL;
+static shmem_request_hook_type prev_shmem_request_hook = NULL;
+
+/* Forward declarations */
+static void pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params);
+static bool evs_oid_in_list(HTAB *hash, Oid oid);
+static void evs_track_hash_ensure_init(void);
+static void evs_track_save_file(void);
+static void evs_track_load_file(void);
+static void evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg);
+static void evs_shmem_request(void);
+
+/* Hash tables for track_databases and track_relations_list (backend-local) */
+static HTAB *evs_track_databases_hash = NULL;
+static HTAB *evs_track_relations_hash = NULL;
+static bool evs_track_hash_initialized = false;
+
+/*
+ * Named LWLock tranche protecting the on-disk track file and serializing
+ * backend-local reloads/saves across concurrent backends.
+ */
+#define EVS_TRACK_TRANCHE_NAME "ext_vacuum_statistics_track"
+static LWLock *evs_track_lock = NULL;
+
+static inline LWLock *
+evs_get_track_lock(void)
+{
+ if (evs_track_lock == NULL)
+ evs_track_lock = &GetNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME)->lock;
+ return evs_track_lock;
+}
+
+/*
+ * objid encoding for relations: (relid << 2) | (type & 3)
+ */
+#define EXTVAC_OBJID(relid, type) (((uint64) (relid)) << 2 | ((type) & 3))
+
+/* Key for relation tracking: (dboid, reloid).
+ * InvalidOid for dboid means it is a cluster object.
+ */
+typedef struct
+{
+ Oid dboid;
+ Oid reloid;
+} EvsTrackRelKey;
+
+/* Shared memory entry for vacuum stats; one per relation or database. */
+typedef struct PgStatShared_ExtVacEntry
+{
+ PgStatShared_Common header;
+ PgStat_VacuumRelationCounts stats;
+} PgStatShared_ExtVacEntry;
+
+/* PgStat kind for per-relation vacuum statistics (tables/indexes) */
+static const PgStat_KindInfo extvac_relation_kind_info = {
+ .name = "ext_vacuum_statistics_relation",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/* PgStat kind for per-database aggregated vacuum statistics */
+static const PgStat_KindInfo extvac_db_kind_info = {
+ .name = "ext_vacuum_statistics_db",
+ .fixed_amount = false,
+ .accessed_across_databases = true,
+ .write_to_file = true,
+ .track_entry_count = true,
+ .shared_size = sizeof(PgStatShared_ExtVacEntry),
+ .shared_data_off = offsetof(PgStatShared_ExtVacEntry, stats),
+ .shared_data_len = sizeof(PgStat_VacuumRelationCounts),
+ .pending_size = 0,
+ .flush_pending_cb = NULL,
+};
+
+/*
+ * Accumulate a single counter only if its category is enabled in
+ * evs_collect_mask. Parentheses around every argument: the macro is invoked
+ * from expression contexts and with expressions as the destination pointer.
+ */
+#define ACCUM_IF(dst, src, field, cat) \
+ do { \
+ if ((evs_collect_mask) & (cat)) \
+ ((dst))->field += ((src))->field; \
+ } while (0)
+
+static inline void
+pgstat_accumulate_common(PgStat_CommonCounts * dst, const PgStat_CommonCounts * src)
+{
+ ACCUM_IF(dst, src, total_blks_read, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_dirtied, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, total_blks_written, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_fetched, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blks_hit, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_read_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, blk_write_time, EVS_COLLECT_BUFFERS);
+ ACCUM_IF(dst, src, delay_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, total_time, EVS_COLLECT_TIMING);
+ ACCUM_IF(dst, src, wal_records, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_fpi, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wal_bytes, EVS_COLLECT_WAL);
+ ACCUM_IF(dst, src, wraparound_failsafe_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, interrupts_count, EVS_COLLECT_GENERAL);
+ ACCUM_IF(dst, src, tuples_deleted, EVS_COLLECT_GENERAL);
+}
+
+static inline void
+pgstat_accumulate_extvac_stats(PgStat_VacuumRelationCounts * dst,
+ const PgStat_VacuumRelationCounts * src)
+{
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type != PGSTAT_EXTVAC_INVALID && src->type != PGSTAT_EXTVAC_DB);
+ Assert(src->type == dst->type);
+
+ pgstat_accumulate_common(&dst->common, &src->common);
+
+ if (dst->type == PGSTAT_EXTVAC_TABLE &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->table.pages_scanned += src->table.pages_scanned;
+ dst->table.pages_removed += src->table.pages_removed;
+ dst->table.tuples_frozen += src->table.tuples_frozen;
+ dst->table.recently_dead_tuples += src->table.recently_dead_tuples;
+ dst->table.vm_new_frozen_pages += src->table.vm_new_frozen_pages;
+ dst->table.vm_new_visible_pages += src->table.vm_new_visible_pages;
+ dst->table.vm_new_visible_frozen_pages += src->table.vm_new_visible_frozen_pages;
+ dst->table.missed_dead_pages += src->table.missed_dead_pages;
+ dst->table.missed_dead_tuples += src->table.missed_dead_tuples;
+ dst->table.index_vacuum_count += src->table.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX &&
+ (evs_collect_mask & EVS_COLLECT_GENERAL) != 0)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ }
+}
+
+/*
+ * GUC check hooks: validate the string and compute the bitmask into *extra.
+ * Rejecting unknown values here prevents silent fall-through to "all".
+ */
+static bool
+evs_track_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_TRACK_RELATIONS | EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "databases") == 0)
+ *bits = EVS_TRACK_DATABASES;
+ else if (strcmp(*newval, "relations") == 0)
+ *bits = EVS_TRACK_RELATIONS;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"databases\", \"relations\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_assign_hook(const char *newval, void *extra)
+{
+ evs_track_bits = *((int *) extra);
+}
+
+static bool
+evs_track_relations_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *bits;
+
+ if (*newval == NULL)
+ return false;
+
+ bits = (int *) guc_malloc(LOG, sizeof(int));
+ if (!bits)
+ return false;
+
+ if (strcmp(*newval, "all") == 0)
+ *bits = EVS_FILTER_SYSTEM | EVS_FILTER_USER;
+ else if (strcmp(*newval, "system") == 0)
+ *bits = EVS_FILTER_SYSTEM;
+ else if (strcmp(*newval, "user") == 0)
+ *bits = EVS_FILTER_USER;
+ else
+ {
+ guc_free(bits);
+ GUC_check_errdetail("Allowed values are \"all\", \"system\", \"user\".");
+ return false;
+ }
+ *extra = bits;
+ return true;
+}
+
+static void
+evs_track_relations_assign_hook(const char *newval, void *extra)
+{
+ evs_track_relations_bits = *((int *) extra);
+}
+
+/*
+ * Check hook for vacuum_statistics.collect.
+ *
+ * Accepts a comma- or whitespace-separated list of category names
+ * (buffers, wal, general, timing) or the shorthand "all". Computes the
+ * matching bitmask once and stashes it in *extra; the assign hook just
+ * copies it into evs_collect_mask. Unknown tokens are rejected so the
+ * setting cannot silently collapse to the "all" default.
+ */
+static bool
+evs_collect_check_hook(char **newval, void **extra, GucSource source)
+{
+ int *mask;
+ char *copy;
+ char *p;
+ char *tok;
+ int accum = 0;
+ bool saw_all = false;
+
+ if (*newval == NULL)
+ return false;
+
+ mask = (int *) guc_malloc(LOG, sizeof(int));
+ if (!mask)
+ return false;
+
+ /* Empty string means "all", matching the default behavior. */
+ if ((*newval)[0] == '\0')
+ {
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+ }
+
+ copy = pstrdup(*newval);
+ for (p = copy; (tok = strtok(p, " \t,")) != NULL; p = NULL)
+ {
+ if (pg_strcasecmp(tok, "all") == 0)
+ saw_all = true;
+ else if (pg_strcasecmp(tok, "buffers") == 0)
+ accum |= EVS_COLLECT_BUFFERS;
+ else if (pg_strcasecmp(tok, "wal") == 0)
+ accum |= EVS_COLLECT_WAL;
+ else if (pg_strcasecmp(tok, "general") == 0)
+ accum |= EVS_COLLECT_GENERAL;
+ else if (pg_strcasecmp(tok, "timing") == 0)
+ accum |= EVS_COLLECT_TIMING;
+ else
+ {
+ /*
+ * GUC_check_errdetail formats the message immediately, but tok
+ * points into copy; emit the detail first, then free the
+ * scratch buffer so the formatted string is already stashed in
+ * GUC_check_errdetail_string.
+ */
+ GUC_check_errdetail("Unrecognized category \"%s\" in vacuum_statistics.collect; "
+ "allowed values are \"all\", \"buffers\", \"wal\", \"general\", \"timing\".",
+ tok);
+ pfree(copy);
+ guc_free(mask);
+ return false;
+ }
+ }
+ pfree(copy);
+
+ *mask = saw_all ? EVS_COLLECT_ALL : accum;
+ if (*mask == 0)
+ *mask = EVS_COLLECT_ALL;
+ *extra = mask;
+ return true;
+}
+
+static void
+evs_collect_assign_hook(const char *newval, void *extra)
+{
+ evs_collect_mask = *((int *) extra);
+}
+
+void
+_PG_init(void)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics module could be loaded only on startup."),
+ errdetail("Add 'ext_vacuum_statistics' into the shared_preload_libraries list.")));
+
+ DefineCustomBoolVariable("vacuum_statistics.enabled",
+ "Enable extended vacuum statistics collection.",
+ NULL, &evs_enabled, true,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.object_types",
+ "Object types for statistics: 'all', 'databases', 'relations'.",
+ NULL, &evs_track, "all",
+ PGC_SUSET, 0,
+ evs_track_check_hook,
+ evs_track_assign_hook, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.track_relations",
+ "When tracking relations: 'all', 'system', 'user'.",
+ NULL, &evs_track_relations, "all",
+ PGC_SUSET, 0,
+ evs_track_relations_check_hook,
+ evs_track_relations_assign_hook, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_databases_from_list",
+ "If true, track only databases added via add_track_database.",
+ NULL, &evs_track_databases_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("vacuum_statistics.track_relations_from_list",
+ "If true, track only relations added via add_track_relation.",
+ NULL, &evs_track_relations_from_list, false,
+ PGC_SUSET, 0, NULL, NULL, NULL);
+
+ DefineCustomStringVariable("vacuum_statistics.collect",
+ "Statistics categories to collect.",
+ "Comma- or whitespace-separated list of: "
+ "\"buffers\", \"wal\", \"general\", \"timing\"; "
+ "or \"all\" for every category (default).",
+ &evs_collect, "all",
+ PGC_SUSET, 0,
+ evs_collect_check_hook,
+ evs_collect_assign_hook, NULL);
+
+ MarkGUCPrefixReserved(SJ_NODENAME);
+
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_RELATION, &extvac_relation_kind_info);
+ pgstat_register_kind(PGSTAT_KIND_EXTVAC_DB, &extvac_db_kind_info);
+
+ prev_shmem_request_hook = shmem_request_hook;
+ shmem_request_hook = evs_shmem_request;
+
+ prev_report_vacuum_hook = set_report_vacuum_hook;
+ set_report_vacuum_hook = pgstat_report_vacuum_extstats;
+
+ prev_object_access_hook = object_access_hook;
+ object_access_hook = evs_drop_access_hook;
+}
+
+static void
+evs_shmem_request(void)
+{
+ if (prev_shmem_request_hook)
+ prev_shmem_request_hook();
+
+ RequestNamedLWLockTranche(EVS_TRACK_TRANCHE_NAME, 1);
+}
+
+/*
+ * Object access hook: remove dropped objects from track lists.
+ */
+static void
+evs_drop_access_hook(ObjectAccessType access, Oid classId,
+ Oid objectId, int subId, void *arg)
+{
+ if (prev_object_access_hook)
+ (*prev_object_access_hook) (access, classId, objectId, subId, arg);
+
+ if (access == OAT_DROP)
+ {
+ if (classId == RelationRelationId && subId == 0)
+ {
+ char relkind = get_rel_relkind(objectId);
+ EvsTrackRelKey key;
+ bool found;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_INDEX)
+ {
+ LWLock *lock = evs_get_track_lock();
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ key.dboid = MyDatabaseId;
+ key.reloid = objectId;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ key.dboid = InvalidOid;
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+
+ if (classId == DatabaseRelationId && objectId != InvalidOid)
+ {
+ LWLock *lock = evs_get_track_lock();
+ bool found;
+
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &objectId, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ }
+ }
+}
+
+/*
+ * Storage of track lists in a separate file.
+ *
+ * Stores the lists of database OIDs and (dboid, reloid) pairs used for
+ * selective tracking when track_databases_from_list or track_relations_from_list
+ * is enabled.
+ * Data stores in pg_stat/ext_vacuum_statistics_track.oid
+ */
+/*
+ * Initialize the backend-local tracking hashes and load their contents
+ * from the on-disk file.
+ *
+ * The hashes are per-backend, so no lock is needed to protect them from
+ * other processes; however, another backend may be concurrently rewriting
+ * the track file, so we take a shared lock for the file read.
+ */
+static void
+evs_track_hash_ensure_init(void)
+{
+ HASHCTL ctl;
+ LWLock *lock;
+ bool need_load;
+
+ if (evs_track_hash_initialized)
+ return;
+
+ lock = evs_get_track_lock();
+
+ if (evs_track_databases_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_databases_hash =
+ hash_create("ext_vacuum_statistics track databases",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ if (evs_track_relations_hash == NULL)
+ {
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(EvsTrackRelKey);
+ ctl.entrysize = sizeof(EvsTrackRelKey);
+ ctl.hcxt = TopMemoryContext;
+ evs_track_relations_hash =
+ hash_create("ext_vacuum_statistics track relations",
+ 64, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ need_load = !LWLockHeldByMe(lock);
+ if (need_load)
+ LWLockAcquire(lock, LW_SHARED);
+ PG_TRY();
+ {
+ evs_track_load_file();
+ evs_track_hash_initialized = true;
+ }
+ PG_FINALLY();
+ {
+ if (need_load)
+ LWLockRelease(lock);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Load track lists from disk into the backend-local hashes.
+ *
+ * Caller must hold evs_track_lock at least in shared mode, since the file
+ * may be concurrently rewritten by another backend.
+ */
+static void
+evs_track_load_file(void)
+{
+ char path[MAXPGPATH];
+ FILE *fp;
+ char buf[MAXPGPATH];
+ bool in_relations = false;
+ Oid oid;
+ EvsTrackRelKey key;
+ bool found;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ fp = AllocateFile(path, "r");
+ if (!fp)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open track file \"%s\": %m", path)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ while (fgets(buf, sizeof(buf), fp))
+ {
+ size_t len = strlen(buf);
+
+ /* Reject unterminated lines (longer than buffer) as corruption. */
+ if (len > 0 && buf[len - 1] != '\n' && !feof(fp))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("line too long in track file \"%s\"", path)));
+
+ if (strncmp(buf, "[databases]", 11) == 0)
+ {
+ in_relations = false;
+ continue;
+ }
+ if (strncmp(buf, "[relations]", 11) == 0)
+ {
+ in_relations = true;
+ continue;
+ }
+ if (in_relations)
+ {
+ if (sscanf(buf, "%u %u", &key.dboid, &key.reloid) == 2)
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ else if (sscanf(buf, "%u", &oid) == 1)
+ {
+ key.dboid = InvalidOid;
+ key.reloid = oid;
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ }
+ }
+ else if (sscanf(buf, "%u", &oid) == 1)
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ }
+
+ if (ferror(fp))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read track file \"%s\": %m", path)));
+ }
+ PG_FINALLY();
+ {
+ FreeFile(fp);
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Atomically rewrite the track file. Caller must hold evs_track_lock
+ * in exclusive mode.
+ */
+static void
+evs_track_save_file(void)
+{
+ char path[MAXPGPATH];
+ char tmppath[MAXPGPATH];
+ FILE *fp;
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+ bool failed = false;
+
+ if (!DataDir || DataDir[0] == '\0' ||
+ !evs_track_databases_hash || !evs_track_relations_hash)
+ return;
+
+ snprintf(path, sizeof(path), "%s/%s", DataDir, EVS_TRACK_FILENAME);
+ snprintf(tmppath, sizeof(tmppath), "%s.tmp", path);
+
+ fp = AllocateFile(tmppath, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create track file \"%s\": %m", tmppath)));
+ return;
+ }
+
+ PG_TRY();
+ {
+ if (fputs("[databases]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ if (fprintf(fp, "%u\n", *entry) < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fputs("[relations]\n", fp) == EOF)
+ failed = true;
+
+ if (!failed)
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ int rc;
+
+ if (OidIsValid(rel_entry->dboid))
+ rc = fprintf(fp, "%u %u\n", rel_entry->dboid, rel_entry->reloid);
+ else
+ rc = fprintf(fp, "0 %u\n", rel_entry->reloid);
+ if (rc < 0)
+ {
+ hash_seq_term(&status);
+ failed = true;
+ break;
+ }
+ }
+ }
+
+ if (!failed && fflush(fp) != 0)
+ failed = true;
+
+ if (!failed)
+ {
+ int fd = fileno(fp);
+
+ if (fd >= 0 && pg_fsync(fd) != 0)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not fsync track file \"%s\": %m",
+ tmppath)));
+ }
+ }
+ PG_CATCH();
+ {
+ FreeFile(fp);
+ (void) unlink(tmppath);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+
+ if (FreeFile(fp) != 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close track file \"%s\": %m", tmppath)));
+ failed = true;
+ }
+
+ if (failed)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write track file \"%s\": %m", tmppath)));
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ return;
+ }
+
+ if (durable_rename(tmppath, path, LOG) != 0)
+ {
+ if (unlink(tmppath) != 0 && errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not unlink \"%s\": %m", tmppath)));
+ }
+}
+
+/*
+ * Check if OID is in the given hash
+ */
+static bool
+evs_oid_in_list(HTAB *hash, Oid oid)
+{
+ if (!hash)
+ return false;
+ if (hash_get_num_entries(hash) == 0)
+ return false;
+ return hash_search(hash, &oid, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Check if (dboid, relid) is in track_relations list.
+ */
+static bool
+evs_rel_in_list(Oid dboid, Oid relid)
+{
+ EvsTrackRelKey key;
+
+ if (!evs_track_relations_hash)
+ return false;
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ return false;
+ key.dboid = dboid;
+ key.reloid = relid;
+ if (hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL)
+ return true;
+ key.dboid = InvalidOid;
+ return hash_search(evs_track_relations_hash, &key, HASH_FIND, NULL) != NULL;
+}
+
+/*
+ * Decide whether to track statistics for relations.
+ * Relation is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_relation_statistics(Oid dboid, Oid relid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if (evs_track_relations_from_list &&
+ !(evs_rel_in_list(dboid, relid) || evs_rel_in_list(InvalidOid, relid)))
+ return false;
+
+ if ((evs_track_bits & EVS_TRACK_RELATIONS) == 0)
+ return false; /* database-only mode */
+ if (evs_track_relations_bits == EVS_FILTER_SYSTEM)
+ return IsCatalogRelationOid(relid);
+ if (evs_track_relations_bits == EVS_FILTER_USER)
+ return !IsCatalogRelationOid(relid);
+ return true;
+}
+
+/*
+ * Decide whether to track statistics for databases.
+ * Database statistics is tracked if it is in the track list or a special filter is enabled.
+ */
+static bool
+evs_should_track_database_statistics(Oid dboid)
+{
+ evs_track_hash_ensure_init();
+
+ if (evs_track_databases_from_list &&
+ !evs_oid_in_list(evs_track_databases_hash, dboid))
+ return false;
+ if ((evs_track_bits & EVS_TRACK_DATABASES) == 0)
+ return false; /* relations-only mode */
+ if (evs_track_bits == EVS_TRACK_DATABASES)
+ return true; /* databases-only, accumulate to db */
+ return true;
+}
+
+
+/* Accumulate common counts for database-level stats. */
+static inline void
+pgstat_accumulate_common_for_db(PgStat_CommonCounts * dst,
+ const PgStat_CommonCounts * src)
+{
+ pgstat_accumulate_common(dst, src);
+}
+
+/*
+ * Store incoming vacuum stats into pgstat custom statistics.
+ * store_relation: create/update per-relation entry
+ * store_db: accumulate into database-level entry (dboid, objid=0).
+ * Uses pgstat_get_entry_ref_locked and pgstat_accumulate_* for atomic updates.
+ */
+static void
+extvac_store(Oid dboid, Oid relid, int type,
+ PgStat_VacuumRelationCounts * params,
+ bool store_relation, bool store_db)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_ExtVacEntry *shared;
+ uint64 objid;
+
+ if (!evs_enabled)
+ return;
+
+ if (store_relation)
+ {
+ objid = EXTVAC_OBJID(relid, type);
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = params->type;
+ }
+ pgstat_accumulate_extvac_stats(&shared->stats, params);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+
+ if (store_db)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_EXTVAC_DB, dboid, InvalidOid, false);
+ if (entry_ref)
+ {
+ shared = (PgStatShared_ExtVacEntry *) entry_ref->shared_stats;
+ if (shared->stats.type == PGSTAT_EXTVAC_INVALID)
+ {
+ memset(&shared->stats, 0, sizeof(shared->stats));
+ shared->stats.type = PGSTAT_EXTVAC_DB;
+ }
+ pgstat_accumulate_common_for_db(&shared->stats.common, ¶ms->common);
+ pgstat_unlock_entry(entry_ref);
+ }
+ }
+}
+
+/*
+ * Vacuum report hook: called when vacuum finishes. Filters by track settings,
+ * stores stats per-relation and/or per-database, then chains to previous hook.
+ */
+static void
+pgstat_report_vacuum_extstats(Oid tableoid, bool shared,
+ PgStat_VacuumRelationCounts * params)
+{
+ Oid dboid = shared ? InvalidOid : MyDatabaseId;
+ bool store_relation;
+ bool store_db;
+
+ if (evs_enabled)
+ {
+ store_relation = evs_should_track_relation_statistics(dboid, tableoid);
+ store_db = evs_should_track_database_statistics(dboid);
+
+ if (store_relation || store_db)
+ extvac_store(dboid, tableoid, params->type, params, store_relation, store_db);
+ }
+ if (prev_report_vacuum_hook)
+ prev_report_vacuum_hook(tableoid, shared, params);
+}
+
+/* Reset statistics for a single relation entry. */
+static bool
+extvac_reset_by_relid(Oid dboid, Oid relid, int type)
+{
+ uint64 objid = EXTVAC_OBJID(relid, type);
+
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_RELATION, dboid, objid, 0);
+ return true;
+}
+
+/* Callback for pgstat_reset_matching_entries: match relation entries for given db */
+static bool
+match_extvac_relations_for_db(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_EXTVAC_RELATION &&
+ entry->key.dboid == DatumGetObjectId(match_data);
+}
+
+/*
+ * Reset statistics for a database (aggregate entry) and all its relations.
+ */
+static int64
+extvac_database_reset(Oid dboid)
+{
+ pgstat_reset_matching_entries(match_extvac_relations_for_db,
+ ObjectIdGetDatum(dboid), 0);
+ pgstat_reset_entry(PGSTAT_KIND_EXTVAC_DB, dboid, 0, 0);
+ return 1;
+}
+
+/* Reset all vacuum statistics (both relation and database entries). */
+static int64
+extvac_stat_reset(void)
+{
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_RELATION);
+ pgstat_reset_of_kind(PGSTAT_KIND_EXTVAC_DB);
+ return 0; /* count not available */
+}
+
+PG_FUNCTION_INFO_V1(vacuum_statistics_reset);
+PG_FUNCTION_INFO_V1(extvac_shared_memory_size);
+PG_FUNCTION_INFO_V1(extvac_reset_entry);
+PG_FUNCTION_INFO_V1(extvac_reset_db_entry);
+
+Datum
+vacuum_statistics_reset(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT64(extvac_stat_reset());
+}
+
+Datum
+extvac_reset_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+ Oid relid = PG_GETARG_OID(1);
+ int type = PG_GETARG_INT32(2);
+
+ PG_RETURN_BOOL(extvac_reset_by_relid(dboid, relid, type));
+}
+
+Datum
+extvac_reset_db_entry(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ PG_RETURN_INT64(extvac_database_reset(dboid));
+}
+
+/*
+ * Return total shared memory in bytes used by the extension for vacuum stats.
+ * Used for monitoring and capacity planning: memory grows with the number of
+ * tracked relations and databases.
+ */
+Datum
+extvac_shared_memory_size(PG_FUNCTION_ARGS)
+{
+ uint64 rel_count;
+ uint64 db_count;
+ uint64 total;
+ size_t entry_size = sizeof(PgStatShared_ExtVacEntry);
+
+ rel_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_RELATION);
+ db_count = pgstat_get_entry_count(PGSTAT_KIND_EXTVAC_DB);
+ total = rel_count + db_count;
+
+ PG_RETURN_INT64((int64) (total * entry_size));
+}
+
+/*
+ * Track list management: add/remove database or relation OIDs.
+ * Changes are persisted to pg_stat/ext_vacuum_statistics_track.oid.
+ */
+
+PG_FUNCTION_INFO_V1(evs_add_track_database);
+PG_FUNCTION_INFO_V1(evs_remove_track_database);
+PG_FUNCTION_INFO_V1(evs_add_track_relation);
+PG_FUNCTION_INFO_V1(evs_remove_track_relation);
+
+/*
+ * Mutating track-list entry points: require server-wide privilege, since
+ * the underlying lists steer tracking for every backend.
+ */
+static void
+evs_require_track_privilege(const char *funcname)
+{
+ if (!superuser() && !has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS))
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for function %s", funcname),
+ errhint("Only superusers and members of pg_read_all_stats "
+ "may change the vacuum statistics track list.")));
+}
+
+Datum
+evs_add_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_database(PG_FUNCTION_ARGS)
+{
+ Oid oid = PG_GETARG_OID(0);
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_database");
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_databases_hash, &oid, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+Datum
+evs_add_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("add_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_ENTER, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(!found); /* true if newly added */
+}
+
+Datum
+evs_remove_track_relation(PG_FUNCTION_ARGS)
+{
+ EvsTrackRelKey key;
+ bool found;
+ LWLock *lock;
+
+ evs_require_track_privilege("remove_track_relation");
+ key.dboid = PG_GETARG_OID(0);
+ key.reloid = PG_GETARG_OID(1);
+ lock = evs_get_track_lock();
+ LWLockAcquire(lock, LW_EXCLUSIVE);
+ evs_track_hash_ensure_init();
+ hash_search(evs_track_relations_hash, &key, HASH_REMOVE, &found);
+ evs_track_save_file();
+ LWLockRelease(lock);
+ PG_RETURN_BOOL(found);
+}
+
+/*
+ * Returns the list of database and relation OIDs for which statistics
+ * are collected.
+ */
+PG_FUNCTION_INFO_V1(evs_track_list);
+
+Datum
+evs_track_list(PG_FUNCTION_ARGS)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Datum values[3];
+ bool nulls[3] = {false, false, false};
+ HASH_SEQ_STATUS status;
+ Oid *entry;
+ EvsTrackRelKey *rel_entry;
+
+ if (!rsinfo || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ evs_track_hash_ensure_init();
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ /* Databases */
+ if (hash_get_num_entries(evs_track_databases_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("database");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_databases_hash);
+ while ((entry = (Oid *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("database");
+ values[1] = ObjectIdGetDatum(*entry);
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[2] = false;
+ }
+ }
+
+ /* Relations */
+ if (hash_get_num_entries(evs_track_relations_hash) == 0)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ nulls[1] = true;
+ nulls[2] = true;
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ nulls[1] = false;
+ nulls[2] = false;
+ }
+ else
+ {
+ hash_seq_init(&status, evs_track_relations_hash);
+ while ((rel_entry = (EvsTrackRelKey *) hash_seq_search(&status)) != NULL)
+ {
+ values[0] = CStringGetTextDatum("relation");
+ values[1] = ObjectIdGetDatum(rel_entry->dboid);
+ values[2] = ObjectIdGetDatum(rel_entry->reloid);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return (Datum) 0;
+}
+
+/*
+ * Output vacuum statistics (tables, indexes, or per-database aggregates).
+ */
+#define EXTVAC_COMMON_STAT_COLS 12
+
+static void
+tuplestore_put_common(PgStat_CommonCounts * vacuum_ext,
+ Datum *values, bool *nulls, int *i)
+{
+ char buf[256];
+ const int base PG_USED_FOR_ASSERTS_ONLY = *i;
+
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_read);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_hit);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_dirtied);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->total_blks_written);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_records);
+ values[(*i)++] = Int64GetDatum(vacuum_ext->wal_fpi);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, vacuum_ext->wal_bytes);
+ values[(*i)++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_read_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->blk_write_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->delay_time);
+ values[(*i)++] = Float8GetDatum(vacuum_ext->total_time);
+ values[(*i)++] = Int32GetDatum(vacuum_ext->wraparound_failsafe_count);
+ Assert((*i - base) == EXTVAC_COMMON_STAT_COLS);
+}
+
+#define EXTVAC_HEAP_STAT_COLS 26
+#define EXTVAC_IDX_STAT_COLS 17
+#define EXTVAC_MAX_STAT_COLS Max(EXTVAC_HEAP_STAT_COLS, EXTVAC_IDX_STAT_COLS)
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_VacuumRelationCounts * vacuum_ext)
+{
+ Datum values[EXTVAC_MAX_STAT_COLS];
+ bool nulls[EXTVAC_MAX_STAT_COLS];
+ int i = 0;
+
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(relid);
+
+ tuplestore_put_common(&vacuum_ext->common, values, nulls, &i);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_fetched - vacuum_ext->common.blks_hit);
+ values[i++] = Int64GetDatum(vacuum_ext->common.blks_hit);
+
+ if (vacuum_ext->type == PGSTAT_EXTVAC_TABLE)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_scanned);
+ values[i++] = Int64GetDatum(vacuum_ext->table.pages_removed);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.vm_new_visible_frozen_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.tuples_frozen);
+ values[i++] = Int64GetDatum(vacuum_ext->table.recently_dead_tuples);
+ values[i++] = Int64GetDatum(vacuum_ext->table.index_vacuum_count);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_pages);
+ values[i++] = Int64GetDatum(vacuum_ext->table.missed_dead_tuples);
+ }
+ else if (vacuum_ext->type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(vacuum_ext->common.tuples_deleted);
+ values[i++] = Int64GetDatum(vacuum_ext->index.pages_deleted);
+ }
+
+ Assert(i == ((vacuum_ext->type == PGSTAT_EXTVAC_TABLE) ? EXTVAC_HEAP_STAT_COLS : EXTVAC_IDX_STAT_COLS));
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, int type)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ext_vacuum_statistics: materialize mode required")));
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "ext_vacuum_statistics: return type must be a row type");
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_TABLE)
+ {
+ Oid relid = PG_GETARG_OID(1);
+ PgStat_VacuumRelationCounts *stats;
+
+ if (!OidIsValid(relid))
+ return (Datum) 0;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, dbid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (!stats)
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_RELATION, InvalidOid,
+ EXTVAC_OBJID(relid, type), NULL);
+
+ if (stats && stats->type == type)
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, stats);
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ if (OidIsValid(dbid))
+ {
+#define EXTVAC_DB_STAT_COLS 14
+ Datum values[EXTVAC_DB_STAT_COLS];
+ bool nulls[EXTVAC_DB_STAT_COLS];
+ int i = 0;
+ PgStat_VacuumRelationCounts *stats;
+
+ stats = (PgStat_VacuumRelationCounts *)
+ pgstat_fetch_entry(PGSTAT_KIND_EXTVAC_DB, dbid,
+ InvalidOid, NULL);
+ if (stats && stats->type == PGSTAT_EXTVAC_DB)
+ {
+ memset(nulls, 0, sizeof(nulls));
+ values[i++] = ObjectIdGetDatum(dbid);
+ tuplestore_put_common(&stats->common, values, nulls, &i);
+ values[i++] = Int32GetDatum(stats->common.interrupts_count);
+ Assert(i == EXTVAC_DB_STAT_COLS);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+ /* invalid dbid: return empty set */
+ }
+ else
+ elog(PANIC, "ext_vacuum_statistics: invalid type %d", type);
+
+ return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_tables);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_indexes);
+PG_FUNCTION_INFO_V1(pg_stats_get_vacuum_database);
+
+Datum
+pg_stats_get_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_TABLE);
+}
+
+Datum
+pg_stats_get_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX);
+}
+
+Datum
+pg_stats_get_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB);
+}
diff --git a/contrib/meson.build b/contrib/meson.build
index ebb7f83d8c5..d7dc0fd07f0 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -26,6 +26,7 @@ subdir('cube')
subdir('dblink')
subdir('dict_int')
subdir('dict_xsyn')
+subdir('ext_vacuum_statistics')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index b9b03654aad..2a38f9042bb 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -141,6 +141,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&dict-int;
&dict-xsyn;
&earthdistance;
+ &extvacuumstatistics;
&file-fdw;
&fuzzystrmatch;
&hstore;
diff --git a/doc/src/sgml/extvacuumstatistics.sgml b/doc/src/sgml/extvacuumstatistics.sgml
new file mode 100644
index 00000000000..75eb4691c4d
--- /dev/null
+++ b/doc/src/sgml/extvacuumstatistics.sgml
@@ -0,0 +1,502 @@
+<!-- doc/src/sgml/extvacuumstatistics.sgml -->
+
+<sect1 id="extvacuumstatistics" xreflabel="ext_vacuum_statistics">
+ <title>ext_vacuum_statistics — extended vacuum statistics</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <primary>ext_vacuum_statistics</primary>
+ </indexterm>
+
+ <para>
+ The <filename>ext_vacuum_statistics</filename> module provides
+ extended per-table, per-index, and per-database vacuum statistics
+ (buffer I/O, WAL, general, timing) via views in the
+ <literal>ext_vacuum_statistics</literal> schema.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>ext_vacuum_statistics</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it registers a vacuum hook at
+ server startup. This means that a server restart is needed to add or remove
+ the module. After installation, run
+ <command>CREATE EXTENSION ext_vacuum_statistics</command> in each database
+ where you want to use it.
+ </para>
+
+ <para>
+ When active, the module provides views
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>,
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>, and
+ <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>,
+ plus functions to reset statistics and manage tracking.
+ </para>
+
+ <para>
+ Each tracked object (one table, one index, or one database) uses
+ approximately 232 bytes of shared memory on Linux x86_64 (e.g. Ubuntu):
+ common stats (buffers, WAL, timing) plus header and LWLock ~144 bytes;
+ type + union ~88 bytes (the union holds table-specific or index-specific
+ fields; the allocated size is the same for both). The exact size depends on the platform. Call
+ <function>ext_vacuum_statistics.shared_memory_size()</function> to get
+ the total shared memory used by the extension. The extension's GUCs allow controlling memory by limiting
+ which objects are tracked:
+ <varname>vacuum_statistics.object_types</varname>,
+ <varname>vacuum_statistics.track_relations</varname>, and
+ <varname>track_*_from_list</varname>.
+ Example: a database with 1000 tables and 2000 indexes uses about 700 KB
+ on Ubuntu ((1000 + 2000 + 1) × 232 bytes).
+ </para>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-tables">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_tables</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname>
+ contains one row for each table in the current database (including TOAST
+ tables), showing statistics about vacuuming that specific table. The columns
+ are shown in <xref linkend="extvacuumstatistics-pg-stats-vacuum-tables-columns"/>.
+ </para>
+
+ <table id="extvacuumstatistics-pg-stats-vacuum-tables-columns">
+ <title><structname>ext_vacuum_statistics.pg_stats_vacuum_tables</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of a table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>schema</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the schema this table is in
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database containing this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks read by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times database blocks were found in the buffer cache by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_dirtied</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks dirtied by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_blks_written</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of database blocks written by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_records</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL records generated by vacuum operations performed on this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_fpi</structfield> <type>int8</type>
+ </para>
+ <para>
+ Total number of WAL full page images generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wal_bytes</structfield> <type>numeric</type>
+ </para>
+ <para>
+ Total amount of WAL bytes generated by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_read_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent reading blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>blk_write_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent writing blocks by vacuum operations, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>delay_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Time spent in vacuum delay points, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_time</structfield> <type>float8</type>
+ </para>
+ <para>
+ Total time of vacuuming this table, in milliseconds
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>wraparound_failsafe_count</structfield> <type>int4</type>
+ </para>
+ <para>
+ Number of times vacuum was run to prevent a wraparound problem
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_read</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of blocks vacuum operations read from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>rel_blks_hit</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times blocks of this table were found in the buffer cache by vacuum
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_deleted</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples vacuum operations deleted from this table
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_scanned</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages examined by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pages_removed</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages removed from physical storage by vacuum operations
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>vm_new_visible_frozen_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages newly set all-visible and all-frozen by vacuum in the visibility map
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>tuples_frozen</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of tuples that vacuum operations marked as frozen
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>recently_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of dead tuples left due to visibility in transactions
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_vacuum_count</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of times indexes on this table were vacuumed
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_pages</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of pages that had at least one missed dead tuple
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>missed_dead_tuples</structfield> <type>int8</type>
+ </para>
+ <para>
+ Number of fully DEAD tuples that could not be pruned due to failure to acquire a cleanup lock
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-indexes">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_indexes</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_indexes</structname>
+ contains one row for each index in the current database, showing statistics
+ about vacuuming that specific index. Columns include
+ <structfield>indexrelid</structfield>, <structfield>schema</structfield>,
+ <structfield>indexrelname</structfield>, <structfield>dbname</structfield>,
+ buffer I/O (<structfield>total_blks_read</structfield>,
+ <structfield>total_blks_hit</structfield>, etc.), WAL
+ (<structfield>wal_records</structfield>, <structfield>wal_fpi</structfield>,
+ <structfield>wal_bytes</structfield>), timing
+ (<structfield>blk_read_time</structfield>, <structfield>blk_write_time</structfield>,
+ <structfield>delay_time</structfield>, <structfield>total_time</structfield>),
+ and <structfield>tuples_deleted</structfield>, <structfield>pages_deleted</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-pg-stats-vacuum-database">
+ <title>The <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname> View</title>
+
+ <indexterm zone="extvacuumstatistics">
+ <secondary>pg_stats_vacuum_database</secondary>
+ </indexterm>
+
+ <para>
+ The view <structname>ext_vacuum_statistics.pg_stats_vacuum_database</structname>
+ contains one row for each database in the cluster, showing aggregate vacuum
+ statistics for that database. Columns include
+ <structfield>dboid</structfield>, <structfield>dbname</structfield>,
+ <structfield>db_blks_read</structfield>, <structfield>db_blks_hit</structfield>,
+ <structfield>db_blks_dirtied</structfield>, <structfield>db_blks_written</structfield>,
+ WAL stats (<structfield>db_wal_records</structfield>,
+ <structfield>db_wal_fpi</structfield>, <structfield>db_wal_bytes</structfield>),
+ timing (<structfield>db_blk_read_time</structfield>,
+ <structfield>db_blk_write_time</structfield>, <structfield>db_delay_time</structfield>,
+ <structfield>db_total_time</structfield>),
+ <structfield>db_wraparound_failsafe_count</structfield>, and
+ <structfield>interrupts_count</structfield>.
+ </para>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-functions">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.shared_memory_size()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the total shared memory in bytes used by the extension for
+ vacuum statistics (relations plus databases).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.vacuum_statistics_reset()</function>
+ <returnvalue>bigint</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Resets all vacuum statistics. Returns the number of entries reset.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a database OID to the tracking list (persisted to
+ <filename>pg_stat/ext_vacuum_statistics_track.oid</filename>).
+ Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_database(dboid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a database OID from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.add_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Adds a (database, relation) OID pair to the tracking list. Returns true if newly added.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.remove_track_relation(dboid oid, reloid oid)</function>
+ <returnvalue>boolean</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Removes a (database, relation) pair from the tracking list. Returns true if found and removed.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <function>ext_vacuum_statistics.track_list()</function>
+ <returnvalue>TABLE(track_kind text, dboid oid, reloid oid)</returnvalue>
+ </term>
+ <listitem>
+ <para>
+ Returns the list of database and relation OIDs for which vacuum statistics
+ are collected. When <structfield>dboid</structfield> or
+ <structfield>reloid</structfield> is NULL, statistics are collected for all.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="extvacuumstatistics-configuration">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><varname>vacuum_statistics.enabled</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Enables extended vacuum statistics collection. Default: <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.object_types</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ Object types for statistics: <literal>all</literal>, <literal>databases</literal>, or
+ <literal>relations</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ When tracking relations: <literal>all</literal>, <literal>system</literal>, or
+ <literal>user</literal>. Default: <literal>all</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_databases_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only databases added via <function>add_track_database</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><varname>vacuum_statistics.track_relations_from_list</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ If on, track only relations added via <function>add_track_relation</function>.
+ Default: <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+</sect1>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 25a85082759..85d721467c0 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -133,6 +133,7 @@
<!ENTITY dict-xsyn SYSTEM "dict-xsyn.sgml">
<!ENTITY dummy-seclabel SYSTEM "dummy-seclabel.sgml">
<!ENTITY earthdistance SYSTEM "earthdistance.sgml">
+<!ENTITY extvacuumstatistics SYSTEM "extvacuumstatistics.sgml">
<!ENTITY file-fdw SYSTEM "file-fdw.sgml">
<!ENTITY fuzzystrmatch SYSTEM "fuzzystrmatch.sgml">
<!ENTITY hstore SYSTEM "hstore.sgml">
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 46+ messages in thread
end of thread, other threads:[~2026-04-28 05:28 UTC | newest]
Thread overview: 46+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-10-28 19:07 Re: Vacuum statistics Alexander Korotkov <[email protected]>
2024-10-28 21:03 ` Jim Nasby <[email protected]>
2024-10-29 12:40 ` Andrei Zubkov <[email protected]>
2024-10-29 22:23 ` Jim Nasby <[email protected]>
2024-11-02 12:22 ` Alena Rybakina <[email protected]>
2024-11-08 19:34 ` Jim Nasby <[email protected]>
2024-11-10 20:09 ` Alena Rybakina <[email protected]>
2024-11-13 00:24 ` Jim Nasby <[email protected]>
2024-11-13 16:21 ` Alena Rybakina <[email protected]>
2024-11-30 04:48 ` Kirill Reshke <[email protected]>
2024-12-02 20:59 ` Alena Rybakina <[email protected]>
2024-12-02 08:27 ` Alexander Korotkov <[email protected]>
2024-12-02 14:46 ` Ilia Evdokimov <[email protected]>
2024-12-02 21:00 ` Alena Rybakina <[email protected]>
2024-12-19 10:40 ` Alena Rybakina <[email protected]>
2025-01-02 20:12 ` Sami Imseih <[email protected]>
2025-01-02 21:14 ` Jim Nasby <[email protected]>
2025-01-02 22:33 ` Sami Imseih <[email protected]>
2025-01-03 19:08 ` Jim Nasby <[email protected]>
2025-01-03 19:46 ` Sami Imseih <[email protected]>
2025-01-03 14:15 ` Greg Sabino Mullane <[email protected]>
2025-01-04 20:37 ` Alena Rybakina <[email protected]>
2025-01-06 02:00 ` Sami Imseih <[email protected]>
2025-01-10 12:04 ` Alena Rybakina <[email protected]>
2025-01-10 15:31 ` Alena Rybakina <[email protected]>
2025-01-13 08:54 ` Andrei Zubkov <[email protected]>
2024-12-02 20:12 ` Alena Rybakina <[email protected]>
2024-12-19 10:37 ` Alena Rybakina <[email protected]>
2026-02-28 22:20 ` Alena Rybakina <[email protected]>
2026-03-09 15:46 ` Alena Rybakina <[email protected]>
2026-03-12 12:02 ` Andrei Lepikhov <[email protected]>
2026-03-12 13:28 ` Andrei Lepikhov <[email protected]>
2026-03-12 18:10 ` Alena Rybakina <[email protected]>
2026-03-13 13:04 ` Alena Rybakina <[email protected]>
2026-03-16 08:45 ` Andrei Lepikhov <[email protected]>
2026-03-16 12:11 ` Alena Rybakina <[email protected]>
2026-03-16 14:27 ` Andrey Borodin <[email protected]>
2026-03-30 06:13 ` Alena Rybakina <[email protected]>
2026-04-14 11:10 ` Alena Rybakina <[email protected]>
2026-04-28 02:16 ` Alena Rybakina <[email protected]>
2026-04-28 05:28 ` Alena Rybakina <[email protected]>
2026-03-16 12:07 ` Alena Rybakina <[email protected]>
2026-03-17 15:27 ` Andrei Zubkov <[email protected]>
2026-03-18 14:19 ` Andrei Zubkov <[email protected]>
2025-01-10 14:51 ` Alena Rybakina <[email protected]>
2025-01-13 11:29 ` Alena Rybakina <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox