From: Andrey Borodin <x4mmm@yandex-team.ru>
Message-Id: <71E42FAF-FF92-4EE5-94A5-341EFB891281@yandex-team.ru>
Content-Type: multipart/mixed;
	boundary="Apple-Mail=_1E6975B4-CF08-402E-8131-29B974A8BDCF"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3864.500.181\))
Subject: Re: [BUG] false positive in bt_index_check in case of short 4B
 varlena datum
Date: Sun, 3 May 2026 23:17:19 +0500
In-Reply-To: <c64tbgvkqu6v5y66lkitiy6dd32ksrwz3nysbclqnyztstx2lj@ymw4bqsf7zpv>
Cc: Alexander Korotkov <aekorotkov@gmail.com>,
 Alexander Lakhin <exclusion@gmail.com>,
 Michael Zhilin <m.zhilin@postgrespro.ru>,
 pgsql-bugs@postgresql.org,
 Yura Sokolov <y.sokolov@postgrespro.ru>
To: Andres Freund <andres@anarazel.de>
References: <c6f376bf-9896-111f-ccb1-a19dc1381ad0@gmail.com>
 <C8DD7CB5-CFE8-4AF5-88D7-B9F441BB5A8C@yandex-team.ru>
 <0b535249-a00c-a38a-85f6-d5a38c62dd55@gmail.com>
 <CAPpHfdsAs-B7O2_=jGbF+BQzuW3kGSboY9CcxzEOWUgxxZCr5Q@mail.gmail.com>
 <e3a78e44-ec6c-5924-7832-63de3aad3e9b@gmail.com>
 <CAPpHfdtZxhgjZTpaLd--dCxzUOL6tN+aAZPBeTsGKHLkMQAZpw@mail.gmail.com>
 <CAPpHfdu412Z+jh5Oyc1yzWFM0+52h0kSGQKV4=xQ0aT0UeY2BA@mail.gmail.com>
 <CAPpHfdspqdzVdEU2qHMibKn8OwVuLoZZYvKm-m5Ffqy2aMXgGQ@mail.gmail.com>
 <7ckc7oka4bvafkf5bwlqs6ygrhlsbhz25ppozfch7zbuxcx3rf@e4pr4oqenalc>
 <DA0695AB-4A40-4272-BE64-D22F81F7713B@yandex-team.ru>
 <c64tbgvkqu6v5y66lkitiy6dd32ksrwz3nysbclqnyztstx2lj@ymw4bqsf7zpv>
Archived-At: <https://www.postgresql.org/message-id/71E42FAF-FF92-4EE5-94A5-341EFB891281%40yandex-team.ru>
Precedence: bulk


--Apple-Mail=_1E6975B4-CF08-402E-8131-29B974A8BDCF
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii


> On 2 May 2026, at 00:41, Andres Freund <andres@anarazel.de> wrote:
>=20
> This is checking (as you noted) !VARATT_IS_EXTENDED, whereas the
> bt_normalize_tuple() code is checking !VARATT_IS_COMPRESSED.
>=20
> VARATT_IS_EXTENDED() will return true for short varlenas (because it's =
not a
> standard 4 byte uncompressed varlena), whereas VARATT_IS_COMPRESSED() =
will
> return false for a short varlena (since it's not compressed).
>=20
> I didn't find other instanes of similar code that uses =
!VARATT_IS_COMPRESSED.

As far as I understand=20

!VARATT_IS_COMPRESSED(DatumGetPointer(normalized[i])) && =
!VARATT_IS_SHORT(DatumGetPointer(normalized[i]))

is exactly

!VARATT_IS_EXTENDED(DatumGetPointer(untoasted_values[i]))

Which is what was proposed in v2 patch. But later was changed to =
!VARATT_IS_COMPRESSED().
As I understood it was done to further strengthen normalization.

So the intent might be that short varatts need normalization in some =
cases. But we have no tests that show such a case.
I tried to build a problematic storage alternation like [0], but =
everything works nicely.

So I propose something in a line with attached patch.


Best regards, Andrey Borodin.

[0] =
https://github.com/postgres/postgres/blob/master/contrib/amcheck/sql/check=
_btree.sql#L170-L172

--Apple-Mail=_1E6975B4-CF08-402E-8131-29B974A8BDCF
Content-Disposition: attachment;
	filename=0001-amcheck-avoid-using-VARSIZE-for-VARATT_IS_SHORT.patch
Content-Type: application/octet-stream;
	x-unix-mode=0644;
	name="0001-amcheck-avoid-using-VARSIZE-for-VARATT_IS_SHORT.patch"
Content-Transfer-Encoding: quoted-printable

=46rom=20276482c76491b1aa18a729dd637342b374dcd7fa=20Mon=20Sep=2017=20=
00:00:00=202001=0AFrom:=20Andrey=20Borodin=20<amborodin@acm.org>=0ADate:=20=
Sun,=203=20May=202026=2023:13:59=20+0500=0ASubject:=20[PATCH]=20amcheck:=20=
avoid=20using=20VARSIZE()=20for=20VARATT_IS_SHORT()=0A=0A---=0A=20=
contrib/amcheck/verify_nbtree.c=20|=202=20+-=0A=20src/include/varatt.h=20=
=20=20=20=20=20=20=20=20=20=20=20|=201=20+=0A=202=20files=20changed,=202=20=
insertions(+),=201=20deletion(-)=0A=0Adiff=20--git=20=
a/contrib/amcheck/verify_nbtree.c=20b/contrib/amcheck/verify_nbtree.c=0A=
index=20b74ab5f7a05..a3394d88a4e=20100644=0A---=20=
a/contrib/amcheck/verify_nbtree.c=0A+++=20=
b/contrib/amcheck/verify_nbtree.c=0A@@=20-2888,7=20+2888,7=20@@=20=
bt_normalize_tuple(BtreeCheckState=20*state,=20IndexTuple=20itup)=0A=20=09=
=09=09=09=09=09=09ItemPointerGetBlockNumber(&(itup->t_tid)),=0A=20=09=09=09=
=09=09=09=09ItemPointerGetOffsetNumber(&(itup->t_tid)),=0A=20=09=09=09=09=
=09=09=09RelationGetRelationName(state->rel))));=0A-=09=09else=20if=20=
(!VARATT_IS_COMPRESSED(DatumGetPointer(normalized[i]))=20&&=0A+=09=09=
else=20if=20(!VARATT_IS_EXTENDED(DatumGetPointer(normalized[i]))=20&&=0A=20=
=09=09=09=09=20VARSIZE(DatumGetPointer(normalized[i]))=20>=20=
TOAST_INDEX_TARGET=20&&=0A=20=09=09=09=09=20(att->attstorage=20=3D=3D=20=
TYPSTORAGE_EXTENDED=20||=0A=20=09=09=09=09=20=20att->attstorage=20=3D=3D=20=
TYPSTORAGE_MAIN))=0Adiff=20--git=20a/src/include/varatt.h=20=
b/src/include/varatt.h=0Aindex=20000bdf33b92..321342cab67=20100644=0A---=20=
a/src/include/varatt.h=0A+++=20b/src/include/varatt.h=0A@@=20-297,6=20=
+297,7=20@@=20typedef=20struct=0A=20static=20inline=20Size=0A=20=
VARSIZE(const=20void=20*PTR)=0A=20{=0A+=09Assert(VARATT_IS_4B(PTR));=0A=20=
=09return=20VARSIZE_4B(PTR);=0A=20}=0A=20=0A--=20=0A2.50.1=20(Apple=20=
Git-155)=0A=0A=

--Apple-Mail=_1E6975B4-CF08-402E-8131-29B974A8BDCF--