pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Andres Freund <andres@anarazel.de>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Fix corruption due to vacuum_defer_cleanup_age underflowing 64bi
Date: Wed, 08 Mar 2023 05:55:41 +0000
Message-ID: <E1pZmm5-002SYd-6R@gemulon.postgresql.org> (raw)

Fix corruption due to vacuum_defer_cleanup_age underflowing 64bit xids

When vacuum_defer_cleanup_age is bigger than the current xid, including the
epoch, the subtraction of vacuum_defer_cleanup_age would lead to a wrapped
around xid. While that normally is not a problem, the subsequent conversion to
a 64bit xid results in a 64bit-xid very far into the future. As that xid is
used as a horizon to detect whether rows versions are old enough to be
removed, that allows removal of rows that are still visible (i.e. corruption).

If vacuum_defer_cleanup_age was never changed from the default, there is no
chance of this bug occurring.

This bug was introduced in dc7420c2c92.  A lesser version of it exists in
12-13, introduced by fb5344c969a, affecting only GiST.

The 12-13 version of the issue can, in rare cases, lead to pages in a gist
index getting recycled too early, potentially causing index entries to be
found multiple times.

The fix is fairly simple - don't allow vacuum_defer_cleanup_age to retreat
further than FirstNormalTransactionId.

Patches to make similar bugs easier to find, by adding asserts to the 64bit
xid infrastructure, have been proposed, but are not suitable for backpatching.

Currently there are no tests for vacuum_defer_cleanup_age. A patch introducing
infrastructure to make writing a test easier has been posted to the list.

Reported-by: Michail Nikolaev <michail.nikolaev@gmail.com>
Reviewed-by: Matthias van de Meent <boekewurm+postgres@gmail.com>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20230108002923.cyoser3ttmt63bfn@awork3.anarazel.de
Backpatch: 12-, but impact/fix is smaller for 12-13

Branch
------
REL_15_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/391f08fd68047881345a69e5a7fff173d8f9c897

Modified Files
--------------
src/backend/storage/ipc/procarray.c | 85 +++++++++++++++++++++++++++++++------
1 file changed, 73 insertions(+), 12 deletions(-)



view thread (5+ messages)  latest in thread

Message-ID: <E1pZmm5-002SYd-6R@gemulon.postgresql.org>
Permalink:  ../E1pZmm5-002SYd-6R@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1pZmm5-002SYd-6R@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: andres@anarazel.de
  Subject: Re: pgsql: Fix corruption due to vacuum_defer_cleanup_age underflowing 64bi
  In-Reply-To: <E1pZmm5-002SYd-6R@gemulon.postgresql.org>
  References: <E1pZmm5-002SYd-6R@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox