pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
To: andrewbille@gmail.com, pgsql-bugs@lists.postgresql.org, Alexander Korotkov <aekorotkov@gmail.com>
Subject: Re: BUG #16619: Amcheck detects corruption in hstore' btree index (ver 2)
Date: Wed, 16 Sep 2020 17:24:00 +0300
Message-ID: <8ede01d3-f09c-248e-9494-63628ff3cf56@postgrespro.ru> (raw)
In-Reply-To: <16619-aaba10f83fdc1c3c@postgresql.org>
References: <16619-aaba10f83fdc1c3c@postgresql.org>

On 16.09.2020 13:34, PG Bug reporting form wrote:
> The following bug has been logged on the website:
>
> Bug reference:      16619
> Logged by:          Andrew Bille
> Email address:      andrewbille@gmail.com
> PostgreSQL version: 13beta3
> Operating system:   Ubuntu-20.04
> Description:
>
> Hello all.
> Index corruption detected after upgrade from v 9.6/10:
>
> mkdir v10
> cd v10
> git clone https://github.com/postgres/postgres.git ./
> git checkout REL_10_STABLE
>   ./configure >/dev/null && make -j8 >/dev/null && make -j8 contrib
>> /dev/null
> echo "SELECT 1;" >> contrib/hstore/sql/hstore.sql
> make check -C contrib/hstore
> cd ..
> mkdir v13
> cd v13
> git clone https://github.com/postgres/postgres.git ./
> git checkout REL_13_STABLE
> ./configure >/dev/null && make -j8 >/dev/null && make -j8 contrib
>> /dev/null
> echo "EXTRA_INSTALL = contrib/amcheck contrib/pageinspect"
>>> contrib/hstore/Makefile
> make check -C contrib/hstore
> export LD_LIBRARY_PATH=tmp_install/usr/local/pgsql/lib
> tmp_install/usr/local/pgsql/bin/initdb -D /tmp/db
> tmp_install/usr/local/pgsql/bin/pg_upgrade -d
> "../v10/contrib/hstore/tmp_check/data" -D /tmp/db -b
> "../v10/tmp_install/usr/local/pgsql/bin" -B
> tmp_install/usr/local/pgsql/bin
> tmp_install/usr/local/pgsql/bin/pg_ctl -D /tmp/db -l l.log start
> tmp_install/usr/local/pgsql/bin/psql -dcontrib_regression -c "create
> extension if not exists amcheck;"
> tmp_install/usr/local/pgsql/bin/psql -dcontrib_regression -c "SELECT
> bt_index_parent_check('hidx', true);"
>
> I get the following error:
> ERROR:  mismatch between parent key and child high key in index "hidx"
> DETAIL:  Target block=3 child block=1 target page lsn=0/16D5758.
>
> (The same procedure with v11/12 doesn't produce the error.)
>
> Thank you.

Good catch.

This check fails at v13 with any upgraded index that have BTREE_VERSION 2.

I think it is an amcheck problem. This check was introduced by commit 
d114cc5387.

The bt_pivot_tuple_identical() function excludes block number from 
comparison:

/*
  * Check if two tuples are binary identical except the block number.  So,
  * this function is capable to compare pivot keys on different levels.
  */
static bool
bt_pivot_tuple_identical(highkey, itup)

But it compares offset number. Which is fine for versions > 2, because 
since covering were introduced we store natts in this field and it is 
always initialized.

  * We store the number of columns present inside pivot tuples by abusing
  * their t_tid offset field, since pivot tuples never need to store a real
  * offset (pivot tuples generally store a downlink in t_tid, though).

For v2 indexes pivot tuple's offset can contain any random number which 
will lead to bt_pivot_tuple_identical() failure.

The fix is pretty simple - only compare data part of the tuples. I think 
we can skip offnum check for all index versions to keep the code simple.

I also noticed a typo in a comment and added minor correction to the patch.

-- 
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachments:

amcheck_old_index_fix_v0.patch (text/x-patch, 1.0K) download diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 060600a556..1a43a08847 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -1629,8 +1629,9 @@ bt_pivot_tuple_identical(IndexTuple itup1, IndexTuple itup2) if (IndexTupleSize(itup1) != IndexTupleSize(itup2)) return false; - if (memcmp(&itup1->t_tid.ip_posid, &itup2->t_tid.ip_posid, - IndexTupleSize(itup1) - offsetof(ItemPointerData, ip_posid)) != 0) + if (memcmp(((char *)itup1 + MAXALIGN(sizeof(IndexTupleData))), + ((char *)itup2 + MAXALIGN(sizeof(IndexTupleData))), + IndexTupleSize(itup1) - MAXALIGN(sizeof(IndexTupleData))) != 0) return false; return true; @@ -1785,7 +1786,7 @@ bt_child_highkey_check(BtreeCheckState *state, rightsplit = P_INCOMPLETE_SPLIT(opaque); /* - * If we visit page with high key, check that it is be equal to the + * If we visit page with high key, check that it is equal to the * target key next to corresponding downlink. */ if (!rightsplit && !P_RIGHTMOST(opaque))

view thread (5+ messages)  latest in thread

Message-ID: <8ede01d3-f09c-248e-9494-63628ff3cf56@postgrespro.ru>
Permalink:  ../8ede01d3-f09c-248e-9494-63628ff3cf56@postgrespro.ru/
Also on:    postgresql.org/message-id/8ede01d3-f09c-248e-9494-63628ff3cf56@postgrespro.ru

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-bugs@postgresql.org
  Cc: a.lubennikova@postgrespro.ru, aekorotkov@gmail.com
  Subject: Re: BUG #16619: Amcheck detects corruption in hstore' btree index (ver 2)
  In-Reply-To: <8ede01d3-f09c-248e-9494-63628ff3cf56@postgrespro.ru>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox