Date: Tue, 23 Nov 2021 17:32:25 -0800
From: Andres Freund <andres@anarazel.de>
To: Peter Geoghegan <pg@bowt.ie>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
	Masahiko Sawada <sawada.mshk@gmail.com>
Subject: Re: Removing more vacuumlazy.c special cases, relfrozenxid
 optimizations
Message-ID: <20211124013225.d67t32hkcbbbsjjc@alap3.anarazel.de>
References: 
 <CAH2-Wznp=c=Opj8Z7RMR3G=ec3_JfGYMN_YvmCEjoPCHzWbx0g@mail.gmail.com>
 <20211122192956.jji6kvzbh5rippla@alap3.anarazel.de>
 <CAH2-WzmWk8zJGfjwyczjohX6CS=8aGoFpivTWg4aF28xCQ0VnA@mail.gmail.com>
 <20211123054902.b5motbm32ftn6xgu@alap3.anarazel.de>
 <CAH2-Wz=xt-Owmk1FEz5hia5dZgjUT5SFhWFdhrjSgBS0=1C6pA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAH2-Wz=xt-Owmk1FEz5hia5dZgjUT5SFhWFdhrjSgBS0=1C6pA@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/20211124013225.d67t32hkcbbbsjjc%40alap3.anarazel.de>
Precedence: bulk

Hi,

On 2021-11-23 17:01:20 -0800, Peter Geoghegan wrote:
> > On reason for my doubt is the following:
> >
> > We can set all-visible on a page without a FPW image (well, as long as hint
> > bits aren't logged). There's a significant difference between needing to WAL
> > log FPIs for every heap page or not, and it's not that rare for data to live
> > shorter than autovacuum_freeze_max_age or that limit never being reached.
> 
> This sounds like an objection to one specific heuristic, and not an
> objection to the general idea.

I understood you to propose that we do not have separate frozen and
all-visible states. Which I think will be problematic, because of scenarios
like the above.


> The only essential part is "opportunistic freezing during vacuum, when the
> cost is clearly very low, and the benefit is probably high". And so it now
> seems you were making a far more limited statement than I first believed.

I'm on board with freezing when we already dirty out the page, and when doing
so doesn't cause an additional FPI. And I don't think I've argued against that
in the past.


> These all-visible (but not all-frozen) heap pages could be considered
> "tenured", since they have survived at least one full VACUUM cycle
> without being unset. So why not also freeze them based on the
> assumption that they'll probably stay that way forever?

Because it's a potentially massive increase in write volume? E.g. if you have
a insert-only workload, and you discard old data by dropping old partitions,
this will often add yet another rewrite, despite your data likely never
getting old enough to need to be frozen.

Given that we often immediately need to start another vacuum just when one
finished, because the vacuum took long enough to reach thresholds of vacuuming
again, I don't think the (auto-)vacuum count is a good proxy.

Maybe you meant this as a more limited concept, i.e. only doing so when the
percentage of all-visible but not all-frozen pages is small?


We could perhaps do better with if we had information about the system-wide
rate of xid throughput and how often / how long past vacuums of a table took.


Greetings,

Andres Freund