Feedback-ID: ia2694551:Fastmail
Date: Wed, 25 May 2022 13:44:54 +0200
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Andres Freund <andres@anarazel.de>
Cc: Andrey Borodin <x4mmm@yandex-team.ru>, Peter Geoghegan <pg@bowt.ie>,
	Michael Paquier <michael@paquier.xyz>,
	=?utf-8?B?0J/QtdGC0YrRgCDQodC70LDQstC+0LI=?= <pet.slavov@gmail.com>,
	PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #17485: Records missing from Primary Key index when doing
 REINDEX INDEX CONCURRENTLY
Message-ID: <202205251144.6t4urostzc3s@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20220524190133.j6ee7zh4f5edt5je@alap3.anarazel.de>
Archived-At: 
 <https://www.postgresql.org/message-id/202205251144.6t4urostzc3s%40alvherre.pgsql>
Precedence: bulk

On 2022-May-24, Andres Freund wrote:

> One might think that could be safe, after all the row is invisible to all
> other backends. The problem is that the validation scan won't see *newer* rows
> either, since they're not visible to the snapshot either. And if the new row
> version is a HOT tuple, it won't have made an index entry on its own. Boom,
> corruption.

Whoa.  My oversight here -- I failed to picture a HOT prune concurrently
with CIC or RIC. 

> Basically snapshots don't work anymore. If PROC_IN_SAFE_IC is set,
> that backend is ignored for the horizon computation for snapshots /
> on-access HOT pruning. Which means that rows that are visible to the
> snapshot can be pruned away.

I wondered if we could have different tuple horizons for HOT pruning
than for vacuum, but looking at ComputeXidHorizons() and users of that,
it looks complicated to adapt.

Another possibility (than reverting the commit altogether) might be to
disable HOT pruning while a process is operating on that relation with
PROC_IN_SAFE_IC.  So CIC/RIC processes are still ignored for VACUUM,
while not creating corrupted indexes.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"Postgres is bloatware by design: it was built to house
 PhD theses." (Joey Hellerstein, SIGMOD annual conference 2002)