From: Noah Misch <noah@leadboat.com>
To: Andres Freund <andres@anarazel.de>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: pgsql-hackers@postgresql.org
Cc: pgsql-release@postgresql.org
Cc: pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURR
Date: Tue, 8 Feb 2022 18:04:03 -0800
Message-ID: <20220209020403.GA3002372@rfd.leadboat.com> (raw)
In-Reply-To: <20220209004347.mnwaoiv5x7dntuvi@alap3.anarazel.de>
References: <E1meSUe-0004WX-H2@gemulon.postgresql.org>
<df7b4c0b-7d92-f03f-75c4-9e08b269a716@enterprisedb.com>
<20220209004347.mnwaoiv5x7dntuvi@alap3.anarazel.de>
On Tue, Feb 08, 2022 at 04:43:47PM -0800, Andres Freund wrote:
> On 2022-02-08 22:13:01 +0100, Tomas Vondra wrote:
> > On 10/24/21 03:40, Noah Misch wrote:
> > > Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURRENTLY.
> > >
> > > CIC and REINDEX CONCURRENTLY assume backends see their catalog changes
> > > no later than each backend's next transaction start. That failed to
> > > hold when a backend absorbed a relevant invalidation in the middle of
> > > running RelationBuildDesc() on the CIC index. Queries that use the
> > > resulting index can silently fail to find rows. Fix this for future
> > > index builds by making RelationBuildDesc() loop until it finishes
> > > without accepting a relevant invalidation. It may be necessary to
> > > reindex to recover from past occurrences; REINDEX CONCURRENTLY suffices.
> > > Back-patch to 9.6 (all supported versions).
> > >
> > > Noah Misch and Andrey Borodin, reviewed (in earlier versions) by Andres
> > > Freund.
> > >
> > > Discussion: https://postgr.es/m/20210730022548.GA1940096@gust.leadboat.com
> > >
> >
> > Unfortunately, this seems to have broken CLOBBER_CACHE_ALWAYS builds. Since
> > this commit, initdb never completes due to infinite retrying over and over
> > (on the first RelationBuildDesc call).
Thanks for the report. I had added the debug_discard arguments of
InvalidateSystemCachesExtended() and RelationCacheInvalidate() to make the new
code survive a CREATE TABLE at debug_discard_caches=5. Apparently that's not
enough for initdb. I'll queue a task to look at it.
It's a good reminder to set wait_timeout on buildfarm animals. (I should take
that advice, too.)
> Ugh. Do we need to do something about WRT the next set of minor releases?
No, given that this code already debuted in the November releases.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-committers@postgresql.org
Cc: noah@leadboat.com, andres@anarazel.de, tomas.vondra@enterprisedb.com, pgsql-hackers@postgresql.org, pgsql-release@postgresql.org
Subject: Re: pgsql: Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURR
In-Reply-To: <20220209020403.GA3002372@rfd.leadboat.com>
References: <E1meSUe-0004WX-H2@gemulon.postgresql.org> <df7b4c0b-7d92-f03f-75c4-9e08b269a716@enterprisedb.com> <20220209004347.mnwaoiv5x7dntuvi@alap3.anarazel.de> <20220209020403.GA3002372@rfd.leadboat.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox