Date: Fri, 25 Feb 2022 11:52:53 -0800
From: Nathan Bossart <nathandbossart@gmail.com>
To: Peter Geoghegan <pg@bowt.ie>
Cc: "Imseih (AWS), Sami" <simseih@amazon.com>,
	"Bossart, Nathan" <bossartn@amazon.com>,
	Masahiko Sawada <sawada.mshk@gmail.com>,
	Justin Pryzby <pryzby@telsasoft.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: Add index scan progress to pg_stat_progress_vacuum
Message-ID: <20220225195253.GA663211@nathanxps13>
References: <6753CC4A-D1D8-477D-9B82-CFE12169DFF5@amazon.com>
 <7A4B3BA8-0768-4463-9B25-25DBECF56B25@amazon.com>
 <96387867-CBAF-4CCF-861E-2F4ACBCBBD68@amazon.com>
 <DC471B48-A815-4326-AC62-3672CDB9B4AB@amazon.com>
 <CC7DB450-A142-43B0-A68F-C6B45DC435F2@amazon.com>
 <441BA1FF-23AA-4CEE-9976-7FB693C30D00@amazon.com>
 <1E7CB510-FB91-4DDB-93E1-A19C05F299F3@amazon.com>
 <20220221190910.GA3740885@nathanxps13>
 <28F7536A-3E62-4A6B-9705-A2377A2D03AE@amazon.com>
 <CAH2-Wzmk7fJuUNVX85AR3eS7MUGait7LrbhP8_8Y74W=m61Hnw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAH2-Wzmk7fJuUNVX85AR3eS7MUGait7LrbhP8_8Y74W=m61Hnw@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/20220225195253.GA663211%40nathanxps13>
Precedence: bulk

On Wed, Feb 23, 2022 at 10:41:36AM -0800, Peter Geoghegan wrote:
> On Wed, Feb 23, 2022 at 10:02 AM Imseih (AWS), Sami <simseih@amazon.com> wrote:
>> If the failsafe kicks in midway through a vacuum, the number indexes_total will not be reset to 0. If INDEX_CLEANUP is turned off, then the value will be 0 at the start of the vacuum.
> 
> The way that this works with num_index_scans is that we "round up"
> when there has been non-zero work in lazy_vacuum_all_indexes(), but
> not if the precheck in lazy_vacuum_all_indexes() fails. That seems
> like a good model to generalize from here. Note that this makes
> INDEX_CLEANUP=off affect num_index_scans in much the same way as a
> VACUUM where the failsafe kicks in very early, during the initial heap
> pass. That is, if the failsafe kicks in before we reach lazy_vacuum()
> for the first time (which is not unlikely), or even in the
> lazy_vacuum_all_indexes() precheck, then num_index_scans will remain
> at 0, just like INDEX_CLEANUP=off.
> 
> The actual failsafe WARNING shows num_index_scans, possibly before it
> gets incremented one last time (by "rounding up"). So it's reasonably
> clear how this all works from that context (assuming that the
> autovacuum logging stuff, which reports num_index_scans, outputs a
> report for a table where the failsafe kicked in).

I am confused.  If failsafe kicks in during the middle of a vacuum, I
(perhaps naively) would expect indexes_total and indexes_processed to go to
zero, and I'd expect to no longer see the "vacuuming indexes" and "cleaning
up indexes" phases.  Otherwise, how would I know that we are now skipping
indexes?  Of course, you won't have any historical context about the index
work done before failsafe kicked in, but IMO it is misleading to still
include it in the progress view.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com