Date: Mon, 19 Apr 2021 20:09:17 -0400
From: Bruce Momjian <bruce@momjian.us>
To: Peter Geoghegan <pg@bowt.ie>
Cc: "Bossart, Nathan" <bossartn@amazon.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>,
	"McAlister, Grant" <grant@amazon.com>,
	"Mlodgenski, Jim" <mlodj@amazon.com>,
	"Nasby, Jim" <nasbyj@amazon.com>, "Hsu, John" <hsuchen@amazon.com>
Subject: Re: partial heap only tuples
Message-ID: <20210420000917.GA30205@momjian.us>
References: <2ECBBCA0-4D8D-4841-8872-4A5BBDC063D2@amazon.com>
 <CAH2-WzmHP7H+81j6BG0BV4swvnV27SwyUeB2W9CcLk7vPMqGBw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAH2-WzmHP7H+81j6BG0BV4swvnV27SwyUeB2W9CcLk7vPMqGBw@mail.gmail.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Archived-At: 
 <https://www.postgresql.org/message-id/20210420000917.GA30205%40momjian.us>
Precedence: bulk

On Sun, Apr 18, 2021 at 04:27:15PM -0700, Peter Geoghegan wrote:
> Everybody tends to talk about HOT as if it works perfectly once you
> make some modest assumptions, such as "there are no long-running
> transactions", and "no UPDATEs will logically modify indexed columns".
> But I tend to doubt that that's truly the case -- I think that there
> are still pathological cases where HOT cannot keep the total table
> size stable in the long run due to subtle effects that eventually
> aggregate into significant issues, like heap fragmentation. Ask Jan
> Wieck about the stability of some of the TPC-C/BenchmarkSQL tables to

...

> We might have successfully fit the successor heap tuple version a
> million times before just by HOT pruning, and yet currently we give up
> just because it didn't work on the one millionth and first occasion --
> don't you think that's kind of silly? We may be able to afford having
> a fallback strategy that is relatively expensive, provided it is
> rarely used. And it might be very effective in the aggregate, despite
> being rarely used -- it might provide us just what we were missing
> before. Just try harder when you run into a problem every once in a
> blue moon!
> 
> A diversity of strategies with fallback behavior is sometimes the best
> strategy. Don't underestimate the contribution of rare and seemingly
> insignificant adverse events. Consider the lifecycle of the data over

That is an intersting point --- we often focus on optimizing frequent
operations, but preventing rare but expensive-in-aggregate events from
happening is also useful.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  If only the physical world exists, free will is an illusion.