Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nHmsg-0005DZ-QE for pgsql-hackers@arkaria.postgresql.org; Wed, 09 Feb 2022 13:19:34 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1nHmsf-0003n9-Nf for pgsql-hackers@arkaria.postgresql.org; Wed, 09 Feb 2022 13:19:33 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nHcL6-0000mn-Fj for pgsql-hackers@lists.postgresql.org; Wed, 09 Feb 2022 02:04:12 +0000 Received: from mail-oi1-x232.google.com ([2607:f8b0:4864:20::232]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1nHcL2-0007Hg-Ja for pgsql-hackers@postgresql.org; Wed, 09 Feb 2022 02:04:12 +0000 Received: by mail-oi1-x232.google.com with SMTP id ay7so1043929oib.8 for ; Tue, 08 Feb 2022 18:04:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leadboat.com; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=S+KUeSHf9SOl96kKxLE3ojJ9Ywzlut+IwjMPudWmyzA=; b=G9TEIYm5GCbp6gCYbqn55vq0pb1FmsPJvWqJ1HArPIihRE0u1Mwqlr/WvjLK3yNZcc Nwc5vfYZP6sU5UMI/W8mJN/AzUiDyOLTs6ZOszlNeMB/VDdYt/GfAlAOK7XIgVtttq1S u968pBXeubfiHTfrA5zGH+AM2klxyDagKJx0M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=S+KUeSHf9SOl96kKxLE3ojJ9Ywzlut+IwjMPudWmyzA=; b=664WYT4is28/4KAy4WJ95bg7NkdYMWzeNmOTBFgBRuhxFn2NSWm/meMYjORAQHPv5R QGbXFiS8L0H/8nSM1Srif7wqB3qoZl6Es1vkwX7SZ1wNV6NPmrgAxQXodVTigz3brjdf 1fPpPtewm3DmWPIY7KGSGPKxEZENbbMv3T/ZB4+h6Audf8Mp9bagXJ7ZthOfMOj5gyBW 6LcNjyC2UG2iLonsWyx1pBmu3qaYcx+AdzrCA6YxZRsnS206jgx+eoDZxutMkdK2mzYT Hvx+EztoKcx7dm7WLdQf7E68EXpb2xO8OakN9IVeLQzmlyMXLDUHVsOC0UXRv6hAyu4S cPBA== X-Gm-Message-State: AOAM533Q6S8jWUhFWLVG69XEnN8YTheyOii0WRlEf531YY7XCTHSsHbQ D8mOhKVTgFv18WBb9kt4IKgCDw== X-Google-Smtp-Source: ABdhPJyNfMNmwm3+/58sVHQ/4IQiViuwmVVMPKCwKSlzUWsvsALh6QkUqwuFpwKua09gxZjQ1YrTjw== X-Received: by 2002:a05:6808:13c8:: with SMTP id d8mr378909oiw.286.1644372246172; Tue, 08 Feb 2022 18:04:06 -0800 (PST) Received: from rfd.leadboat.com ([2600:1702:a20:5750::2e]) by smtp.gmail.com with ESMTPSA id h203sm6412735oif.27.2022.02.08.18.04.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 Feb 2022 18:04:05 -0800 (PST) Date: Tue, 8 Feb 2022 18:04:03 -0800 From: Noah Misch To: Andres Freund Cc: Tomas Vondra , pgsql-hackers@postgresql.org, pgsql-release@postgresql.org, pgsql-committers@lists.postgresql.org Subject: Re: pgsql: Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURR Message-ID: <20220209020403.GA3002372@rfd.leadboat.com> References: <20220209004347.mnwaoiv5x7dntuvi@alap3.anarazel.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220209004347.mnwaoiv5x7dntuvi@alap3.anarazel.de> User-Agent: Mutt/1.5.24 (2015-08-30) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Tue, Feb 08, 2022 at 04:43:47PM -0800, Andres Freund wrote: > On 2022-02-08 22:13:01 +0100, Tomas Vondra wrote: > > On 10/24/21 03:40, Noah Misch wrote: > > > Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURRENTLY. > > > > > > CIC and REINDEX CONCURRENTLY assume backends see their catalog changes > > > no later than each backend's next transaction start. That failed to > > > hold when a backend absorbed a relevant invalidation in the middle of > > > running RelationBuildDesc() on the CIC index. Queries that use the > > > resulting index can silently fail to find rows. Fix this for future > > > index builds by making RelationBuildDesc() loop until it finishes > > > without accepting a relevant invalidation. It may be necessary to > > > reindex to recover from past occurrences; REINDEX CONCURRENTLY suffices. > > > Back-patch to 9.6 (all supported versions). > > > > > > Noah Misch and Andrey Borodin, reviewed (in earlier versions) by Andres > > > Freund. > > > > > > Discussion: https://postgr.es/m/20210730022548.GA1940096@gust.leadboat.com > > > > > > > Unfortunately, this seems to have broken CLOBBER_CACHE_ALWAYS builds. Since > > this commit, initdb never completes due to infinite retrying over and over > > (on the first RelationBuildDesc call). Thanks for the report. I had added the debug_discard arguments of InvalidateSystemCachesExtended() and RelationCacheInvalidate() to make the new code survive a CREATE TABLE at debug_discard_caches=5. Apparently that's not enough for initdb. I'll queue a task to look at it. It's a good reminder to set wait_timeout on buildfarm animals. (I should take that advice, too.) > Ugh. Do we need to do something about WRT the next set of minor releases? No, given that this code already debuted in the November releases.