From: Noah Misch <noah@leadboat.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: Peter Geoghegan <pg@bowt.ie>
Cc: David Rowley <dgrowleyml@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Intermittent buildfarm failures on wrasse
Date: Thu, 14 Apr 2022 19:50:19 -0700
Message-ID: <20220415025019.GA863781@rfd.leadboat.com> (raw)
In-Reply-To: <20220415024515.GA862547@rfd.leadboat.com>
References: <20220414161850.ibqkevxcv6a7vdxg@alap3.anarazel.de>
<1521388.1649953580@sss.pgh.pa.us>
<1547912.1649971995@sss.pgh.pa.us>
<CAH2-Wz=DHY2LV7=2aVd=qz6AbQhFK7KJRNL5ZwgB7dyno_gzdg@mail.gmail.com>
<1553951.1649975018@sss.pgh.pa.us>
<CAH2-WzngTjUk_65ESmrMf40WTqasrmWG0gpwnvcN+mXsoXT7qQ@mail.gmail.com>
<CAH2-WznXXkkfE_y9ieaHkEQj31HeMAqJBtJPmDY+EpyVg1QywQ@mail.gmail.com>
<1570894.1649986347@sss.pgh.pa.us>
<20220415015249.kfsqxujrf2bn2wz6@alap3.anarazel.de>
<20220415024515.GA862547@rfd.leadboat.com>
On Thu, Apr 14, 2022 at 07:45:15PM -0700, Noah Misch wrote:
> On Thu, Apr 14, 2022 at 06:52:49PM -0700, Andres Freund wrote:
> > On 2022-04-14 21:32:27 -0400, Tom Lane wrote:
> > > Peter Geoghegan <pg@bowt.ie> writes:
> > > > Are you aware of Andres' commit 02fea8fd? That work prevented exactly
> > > > the same set of symptoms (the same index-only scan create_index
> > > > regressions),
> > >
> > > Hm. I'm starting to get the feeling that the real problem here is
> > > we've "optimized" the system to the point where repeatable results
> > > from VACUUM are impossible :-(
> >
> > The synchronous_commit issue is an old one. It might actually be worth
> > addressing it by flushing out pending async commits out instead. It just
> > started to be noticeable when tenk1 load and vacuum were moved closer.
> >
> >
> > What do you think about applying a polished version of what I posted in
> > https://postgr.es/m/20220414164830.63rk5zqsvtqqk7qz%40alap3.anarazel.de
> > ? That'd tell us a bit more about the horizon etc.
>
> No objection.
>
> > It's also interesting that it only happens in the installcheck cases,
> > afaics, not the check ones. Although that might just be because there's
> > more of them...
>
> I suspect the failure is somehow impossible in "check". Yesterday, I cranked
> up the number of locales, so there are now a lot more installcheck. Before
> that, each farm run had one "check" and two "installcheck". Those days saw
> ten installcheck failures, zero check failures.
>
> Like Tom, I'm failing to reproduce this outside the buildfarm client. I wrote
> a shell script to closely resemble the buildfarm installcheck sequence, but
> it's lasted a dozen runs without failing.
But 24s after that email, it did reproduce the problem. Same symptoms as the
last buildfarm runs, including visfrac=0. I'm attaching my script. (It has
various references to my home directory, so it's not self-contained.)
Attachments:
[application/x-sh] repro-wrasse-vis.sh (1.3K, ../20220415025019.GA863781@rfd.leadboat.com/2-repro-wrasse-vis.sh)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: noah@leadboat.com, tgl@sss.pgh.pa.us, andres@anarazel.de, pg@bowt.ie, dgrowleyml@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: Intermittent buildfarm failures on wrasse
In-Reply-To: <20220415025019.GA863781@rfd.leadboat.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox