Date: Fri, 8 Nov 2024 17:02:36 -0800
From: Noah Misch <noah@leadboat.com>
To: Alexander Lakhin <exclusion@gmail.com>,
	Nitin Motiani <nitinmotiani@google.com>,
	Andres Freund <andres@anarazel.de>, pgsql-hackers@postgresql.org
Subject: Re: Inval reliability, especially for inplace updates
Message-ID: <20241109010236.93.nmisch@google.com>
References: 
 <CAH5HC94r2ZKy_MhNaPUjn7NHkOFgbyBAgV2jaYSRt35BNWuOAA@mail.gmail.com>
 <CAH5HC95rn4gON8zNrFXFY-Cn4z0NZuDU_1TDwKnafX_25_C0UA@mail.gmail.com>
 <20241024025357.a8.nmisch@google.com>
 <CAH5HC97fbvr63wJO23k6N0YmQh8BK3QE8ch3cQC=dWJJBxaCjw@mail.gmail.com>
 <20241031013924.cd.nmisch@google.com>
 <10ec0bc3-5933-1189-6bb8-5dec4114558e@gmail.com>
 <20241031200139.b4@rfd.leadboat.com>
 <20241101042052.4c.nmisch@google.com>
 <20241101233829.9f.nmisch@google.com>
 <20241103182925.93.nmisch@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20241103182925.93.nmisch@google.com>
User-Agent: Mutt/2.2.12 (2023-09-09)
Archived-At: 
 <https://www.postgresql.org/message-id/20241109010236.93.nmisch%40google.com>
Precedence: bulk

On Sun, Nov 03, 2024 at 10:29:25AM -0800, Noah Misch wrote:
> Pushed as 0bada39.
> 
> Buildfarm member hornet REL_15_STABLE was in the same hang.  Other buildfarm
> runs 2024-10-25T13:51:02Z - 2024-11-02T16:04:56Z may hang the same way.  It's
> early to make a comprehensive list of hung buildfarm members, since many
> reported infrequently even before this period.  I'll wait a week or two and
> then contact the likely-hung member owners.  I regret the damage.

Buildfarm members plover and (less likely) sevengill might be hung in this
way.  Owners (bcc'd): if convenient, please check whether a buildfarm run that
started before 2024-11-02T16:04:56Z is still ongoing.  If it is, please use
"kill -QUIT" on one of the long-running autovacuum workers.

If it's the same hang, one autovacuum worker will have stack frames like:

#37 0x0000555e8a4fabcb in CacheInvalidateHeapTupleCommon (prepare_callback=0x555e8a4fa320 <PrepareInplaceInvalidationState>, newtuple=newtuple@entry=0x0, 
    tuple=tuple@entry=0x7f168e9a74e8, relation=relation@entry=0x7f168f1738c8) at inval.c:1504
#38 0x0000555e8a094f7c in heap_inplace_update_and_unlock (relation=0x7f168f1738c8, oldtup=0x555eb277ea58, tuple=0x7f168e9a74e8, buffer=3) at heapam.c:6354