pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #16619: Amcheck detects corruption in hstore' btree index (ver 2)
5+ messages / 4 participants
[nested] [flat]
Commits from this thread:
2020-09-16 c287f58586 aac80bfcdd Fix amcheck child check pg_upgrade bug.
* BUG #16619: Amcheck detects corruption in hstore' btree index (ver 2)
@ 2020-09-16 10:34 PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 5+ messages in thread
From: PG Bug reporting form @ 2020-09-16 10:34 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: andrewbille@gmail.com
The following bug has been logged on the website:
Bug reference: 16619
Logged by: Andrew Bille
Email address: andrewbille@gmail.com
PostgreSQL version: 13beta3
Operating system: Ubuntu-20.04
Description:
Hello all.
Index corruption detected after upgrade from v 9.6/10:
mkdir v10
cd v10
git clone https://github.com/postgres/postgres.git ./
git checkout REL_10_STABLE
./configure >/dev/null && make -j8 >/dev/null && make -j8 contrib
>/dev/null
echo "SELECT 1;" >> contrib/hstore/sql/hstore.sql
make check -C contrib/hstore
cd ..
mkdir v13
cd v13
git clone https://github.com/postgres/postgres.git ./
git checkout REL_13_STABLE
./configure >/dev/null && make -j8 >/dev/null && make -j8 contrib
>/dev/null
echo "EXTRA_INSTALL = contrib/amcheck contrib/pageinspect"
>>contrib/hstore/Makefile
make check -C contrib/hstore
export LD_LIBRARY_PATH=tmp_install/usr/local/pgsql/lib
tmp_install/usr/local/pgsql/bin/initdb -D /tmp/db
tmp_install/usr/local/pgsql/bin/pg_upgrade -d
"../v10/contrib/hstore/tmp_check/data" -D /tmp/db -b
"../v10/tmp_install/usr/local/pgsql/bin" -B
tmp_install/usr/local/pgsql/bin
tmp_install/usr/local/pgsql/bin/pg_ctl -D /tmp/db -l l.log start
tmp_install/usr/local/pgsql/bin/psql -dcontrib_regression -c "create
extension if not exists amcheck;"
tmp_install/usr/local/pgsql/bin/psql -dcontrib_regression -c "SELECT
bt_index_parent_check('hidx', true);"
I get the following error:
ERROR: mismatch between parent key and child high key in index "hidx"
DETAIL: Target block=3 child block=1 target page lsn=0/16D5758.
(The same procedure with v11/12 doesn't produce the error.)
Thank you.
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #16619: Amcheck detects corruption in hstore' btree index (ver 2)
@ 2020-09-16 14:24 Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
parent: PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 5+ messages in thread
From: Anastasia Lubennikova @ 2020-09-16 14:24 UTC (permalink / raw)
To: andrewbille@gmail.com, pgsql-bugs@lists.postgresql.org, Alexander Korotkov <aekorotkov@gmail.com>
On 16.09.2020 13:34, PG Bug reporting form wrote:
> The following bug has been logged on the website:
>
> Bug reference: 16619
> Logged by: Andrew Bille
> Email address: andrewbille@gmail.com
> PostgreSQL version: 13beta3
> Operating system: Ubuntu-20.04
> Description:
>
> Hello all.
> Index corruption detected after upgrade from v 9.6/10:
>
> mkdir v10
> cd v10
> git clone https://github.com/postgres/postgres.git ./
> git checkout REL_10_STABLE
> ./configure >/dev/null && make -j8 >/dev/null && make -j8 contrib
>> /dev/null
> echo "SELECT 1;" >> contrib/hstore/sql/hstore.sql
> make check -C contrib/hstore
> cd ..
> mkdir v13
> cd v13
> git clone https://github.com/postgres/postgres.git ./
> git checkout REL_13_STABLE
> ./configure >/dev/null && make -j8 >/dev/null && make -j8 contrib
>> /dev/null
> echo "EXTRA_INSTALL = contrib/amcheck contrib/pageinspect"
>>> contrib/hstore/Makefile
> make check -C contrib/hstore
> export LD_LIBRARY_PATH=tmp_install/usr/local/pgsql/lib
> tmp_install/usr/local/pgsql/bin/initdb -D /tmp/db
> tmp_install/usr/local/pgsql/bin/pg_upgrade -d
> "../v10/contrib/hstore/tmp_check/data" -D /tmp/db -b
> "../v10/tmp_install/usr/local/pgsql/bin" -B
> tmp_install/usr/local/pgsql/bin
> tmp_install/usr/local/pgsql/bin/pg_ctl -D /tmp/db -l l.log start
> tmp_install/usr/local/pgsql/bin/psql -dcontrib_regression -c "create
> extension if not exists amcheck;"
> tmp_install/usr/local/pgsql/bin/psql -dcontrib_regression -c "SELECT
> bt_index_parent_check('hidx', true);"
>
> I get the following error:
> ERROR: mismatch between parent key and child high key in index "hidx"
> DETAIL: Target block=3 child block=1 target page lsn=0/16D5758.
>
> (The same procedure with v11/12 doesn't produce the error.)
>
> Thank you.
Good catch.
This check fails at v13 with any upgraded index that have BTREE_VERSION 2.
I think it is an amcheck problem. This check was introduced by commit
d114cc5387.
The bt_pivot_tuple_identical() function excludes block number from
comparison:
/*
* Check if two tuples are binary identical except the block number. So,
* this function is capable to compare pivot keys on different levels.
*/
static bool
bt_pivot_tuple_identical(highkey, itup)
But it compares offset number. Which is fine for versions > 2, because
since covering were introduced we store natts in this field and it is
always initialized.
* We store the number of columns present inside pivot tuples by abusing
* their t_tid offset field, since pivot tuples never need to store a real
* offset (pivot tuples generally store a downlink in t_tid, though).
For v2 indexes pivot tuple's offset can contain any random number which
will lead to bt_pivot_tuple_identical() failure.
The fix is pretty simple - only compare data part of the tuples. I think
we can skip offnum check for all index versions to keep the code simple.
I also noticed a typo in a comment and added minor correction to the patch.
--
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Attachments:
amcheck_old_index_fix_v0.patch (text/x-patch, 1.0K) download
diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c
index 060600a556..1a43a08847 100644
--- a/contrib/amcheck/verify_nbtree.c
+++ b/contrib/amcheck/verify_nbtree.c
@@ -1629,8 +1629,9 @@ bt_pivot_tuple_identical(IndexTuple itup1, IndexTuple itup2)
if (IndexTupleSize(itup1) != IndexTupleSize(itup2))
return false;
- if (memcmp(&itup1->t_tid.ip_posid, &itup2->t_tid.ip_posid,
- IndexTupleSize(itup1) - offsetof(ItemPointerData, ip_posid)) != 0)
+ if (memcmp(((char *)itup1 + MAXALIGN(sizeof(IndexTupleData))),
+ ((char *)itup2 + MAXALIGN(sizeof(IndexTupleData))),
+ IndexTupleSize(itup1) - MAXALIGN(sizeof(IndexTupleData))) != 0)
return false;
return true;
@@ -1785,7 +1786,7 @@ bt_child_highkey_check(BtreeCheckState *state,
rightsplit = P_INCOMPLETE_SPLIT(opaque);
/*
- * If we visit page with high key, check that it is be equal to the
+ * If we visit page with high key, check that it is equal to the
* target key next to corresponding downlink.
*/
if (!rightsplit && !P_RIGHTMOST(opaque))
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #16619: Amcheck detects corruption in hstore' btree index (ver 2)
@ 2020-09-16 17:47 Peter Geoghegan <pg@bowt.ie>
parent: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
0 siblings, 1 reply; 5+ messages in thread
From: Peter Geoghegan @ 2020-09-16 17:47 UTC (permalink / raw)
To: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>; +Cc: andrewbille@gmail.com, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>; Alexander Korotkov <aekorotkov@gmail.com>
On Wed, Sep 16, 2020 at 7:24 AM Anastasia Lubennikova
<a.lubennikova@postgrespro.ru> wrote:
> For v2 indexes pivot tuple's offset can contain any random number which
> will lead to bt_pivot_tuple_identical() failure.
>
> The fix is pretty simple - only compare data part of the tuples. I think
> we can skip offnum check for all index versions to keep the code simple.
I pushed a tweaked version of this patch. It seemed worth preserving
the inclusion of offset number in the bt_pivot_tuple_identical()
memcmp() comparison. The field contains important metadata that really
should match. Also, I made sure to compare t_info field on all indexes
(including pg_upgrade'd indexes) for the same reason.
We already account for heapkeyspace difference in other similar
routines, such as invariant_l_offset(). It's not hard to do this here
as well.
Thanks
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #16619: Amcheck detects corruption in hstore' btree index (ver 2)
@ 2020-09-16 21:55 Alexander Korotkov <aekorotkov@gmail.com>
parent: Peter Geoghegan <pg@bowt.ie>
0 siblings, 1 reply; 5+ messages in thread
From: Alexander Korotkov @ 2020-09-16 21:55 UTC (permalink / raw)
To: Peter Geoghegan <pg@bowt.ie>; +Cc: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>; andrewbille@gmail.com, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Hi, Peter!
On Wed, Sep 16, 2020 at 8:47 PM Peter Geoghegan <pg@bowt.ie> wrote:
> On Wed, Sep 16, 2020 at 7:24 AM Anastasia Lubennikova
> <a.lubennikova@postgrespro.ru> wrote:
> > For v2 indexes pivot tuple's offset can contain any random number which
> > will lead to bt_pivot_tuple_identical() failure.
> >
> > The fix is pretty simple - only compare data part of the tuples. I think
> > we can skip offnum check for all index versions to keep the code simple.
>
> I pushed a tweaked version of this patch. It seemed worth preserving
> the inclusion of offset number in the bt_pivot_tuple_identical()
> memcmp() comparison. The field contains important metadata that really
> should match. Also, I made sure to compare t_info field on all indexes
> (including pg_upgrade'd indexes) for the same reason.
>
> We already account for heapkeyspace difference in other similar
> routines, such as invariant_l_offset(). It's not hard to do this here
> as well.
I was going to tweak this patch in a similar way, but made it in
advance. Your commit looks good to me, thanks!
------
Regards,
Alexander Korotkov
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #16619: Amcheck detects corruption in hstore' btree index (ver 2)
@ 2020-09-16 22:29 Peter Geoghegan <pg@bowt.ie>
parent: Alexander Korotkov <aekorotkov@gmail.com>
0 siblings, 0 replies; 5+ messages in thread
From: Peter Geoghegan @ 2020-09-16 22:29 UTC (permalink / raw)
To: Alexander Korotkov <aekorotkov@gmail.com>; +Cc: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>; andrewbille@gmail.com, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Hi Alexander,
On Wed, Sep 16, 2020 at 2:56 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> I was going to tweak this patch in a similar way, but made it in
> advance. Your commit looks good to me, thanks!
I would have left it to you, but we're pretty close to releasing 13,
and I didn't want to risk missing the deadline.
FWIW, the reason that Andrew did not see the same problem when
pg_upgrade'ing from Postgres 11 - Postgres 13 is that the INCLUDE
index feature simplified _bt_insert_parent() for all indexes (not just
indexes with non-key columns). That function used to set the offset
number of the pivot tuple it inserts into the parent to P_HIKEY (even
though that was always unnecessary and unhelpful -- maybe you remember
our discussion about this exact thing in early 2018). The pre-11
behavior of _bt_insert_parent() broke the assumption that each pivot
tuple's contents (after the downlink/block number) must always have
the same value in the matching pivot tuple one level up.
FWIW, Anastasia was right to say that the INCLUDE index feature
prevented such an inconsistency from arising -- it was v11 that
removed the silly set-P_HIKEY thing (not 12 or 13). However, the only
practical way for amcheck to know for sure that the index can never
have been affected by the P_HIKEY thing is to check that the index is
a heapkeyspace index (i.e. an index built by Postgres 12 or 13).
(Maybe this is obvious, but I thought I'd say it just in case, for the
archives.)
Thanks
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2020-09-16 22:29 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-09-16 10:34 BUG #16619: Amcheck detects corruption in hstore' btree index (ver 2) PG Bug reporting form <noreply@postgresql.org>
2020-09-16 14:24 ` Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
2020-09-16 17:47 ` Peter Geoghegan <pg@bowt.ie>
2020-09-16 21:55 ` Alexander Korotkov <aekorotkov@gmail.com>
2020-09-16 22:29 ` Peter Geoghegan <pg@bowt.ie>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox