MIME-Version: 1.0
References: 
 <AH*AvQCYKhQGVvPWi1GiU4oY.8.1781609375063.Hmail.3020001251@tju.edu.cn>
In-Reply-To: 
 <AH*AvQCYKhQGVvPWi1GiU4oY.8.1781609375063.Hmail.3020001251@tju.edu.cn>
From: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Date: Thu, 18 Jun 2026 00:49:03 +0530
Message-ID: 
 <CAJTYsWWSx+5-kVaivB_znaQCk3TmHrO4QNNM-CD3AN-oOSmG8w@mail.gmail.com>
Subject: Re: Fw:Re: Fw: gbt_var_consistent in
 contrib/btree_gist/btree_utils_var.c
 has internal-node type confusion on the <> strategy,
 bypassing exclusion constraints
To: =?UTF-8?B?546L6LeD5p6X?= <violin0613@tju.edu.cn>
Cc: pgsql-bugs <pgsql-bugs@postgresql.org>
Content-Type: multipart/mixed; boundary="00000000000078a052065477efad"
Archived-At: 
 <https://www.postgresql.org/message-id/CAJTYsWWSx%2B5-kVaivB_znaQCk3TmHrO4QNNM-CD3AN-oOSmG8w%40mail.gmail.com>
Precedence: bulk

--00000000000078a052065477efad
Content-Type: multipart/alternative; boundary="00000000000078a050065477efab"

--00000000000078a050065477efab
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi,

On Tue, 16 Jun 2026 at 16:59, =E7=8E=8B=E8=B7=83=E6=9E=97 <violin0613@tju.e=
du.cn> wrote:

> 0800, =E7=8E=8B=E8=B7=83=E6=9E=97 wrote:
>
> > gbt_var_node_truncate (btree_utils_var.c:214) truncates internal node k=
eys to a common-prefix length. The resulting bytea can have VARSIZE anywher=
e from 4 upward. When the truncated VARSIZE is below 8 and that key reaches=
 bit_cmp via the buggy BtreeGistNotEqual branch, bytelen becomes negative. =
Passed to memcmp as size_t, that is several GB. ASan catches it as negative=
-size-param. A production build without ASan will eventually SEGV when the =
read crosses an unmapped page.
>
> Thanks for the report!

In gbt_var_consistent() (contrib/btree_gist/btree_utils_var.c), the <>
(BtreeGistNotEqual) branch is the only strategy that does not distinguish
leaf from internal pages. It applies the type's equality function directly
to the stored key on both bounds:

  retval =3D !(tinfo->f_eq(query, key->lower, ...) &&
           tinfo->f_eq(query, key->upper, ...));

That seems to be the problem: on internal pages those keys are
truncated (and for bit/varbit the length header is gone too), so
they aren't really valid values to hand to the type's equality
function.  Hence the negative length reaching memcmp().

Would it make sense to just have <> follow the same leaf-vs-internal
pattern as the other strategies; negate equality on a leaf, and
otherwise recurse?  An internal page can always contain some
non-equal value, so I don't think we lose anything by descending.
I put together a small patch along those lines (plus a bit(8) test
that goes through internal pages); it's attached if it's useful.
(PS. The test exercises the path but passes on a plain build; the OOB
read shows up under ASan (negative-size-param).)

I didn't touch the fixed-size path in btree_utils_num.c, since its
internal keys look like they're exact lower/upper values, but
please correct me if I'm missing something there.

Regards,
Ayush

--00000000000078a050065477efab
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">Hi,</div><br><div class=3D"gmail_quote gm=
ail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, 16 Jun 2=
026 at 16:59, =E7=8E=8B=E8=B7=83=E6=9E=97 &lt;<a href=3D"mailto:violin0613@=
tju.edu.cn">violin0613@tju.edu.cn</a>&gt; wrote:<br></div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg=
b(204,204,204);padding-left:1ex"><div style=3D"line-height:1.5;font-size:14=
px;color:rgba(38,42,51,0.9);font-family:&quot;Source Han Sans&quot;">
<div style=3D"font-family:&quot;Source Han Sans&quot;;font-size:14px;line-h=
eight:1.5">0800, =E7=8E=8B=E8=B7=83=E6=9E=97 wrote:</div><div><div id=3D"m_=
-7894241906402760152isForwardContent" style=3D"margin:0px"><pre>&gt; gbt_va=
r_node_truncate (btree_utils_var.c:214) truncates internal node keys to a c=
ommon-prefix length. The resulting bytea can have VARSIZE anywhere from 4 u=
pward. When the truncated VARSIZE is below 8 and that key reaches bit_cmp v=
ia the buggy BtreeGistNotEqual branch, bytelen becomes negative. Passed to =
memcmp as size_t, that is several GB. ASan catches it as negative-size-para=
m. A production build without ASan will eventually SEGV when the read cross=
es an unmapped page.</pre></div></div></div></blockquote><div>Thanks for th=
e report!<br><br>In gbt_var_consistent() (contrib/btree_gist/btree_utils_va=
r.c), the &lt;&gt;<br>(BtreeGistNotEqual) branch is the only strategy that =
does not distinguish<br>leaf from internal pages. It applies the type&#39;s=
 equality function directly<br>to the stored key on both bounds:<br><br>=C2=
=A0 retval =3D !(tinfo-&gt;f_eq(query, key-&gt;lower, ...) &amp;&amp;<br>=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tinfo-&gt;f_eq(query, key-&gt;uppe=
r, ...));<br><br>That seems to be the problem: on internal pages those keys=
 are<br>truncated (and for bit/varbit the length header is gone too), so<br=
>they aren&#39;t really valid values to hand to the type&#39;s equality<br>=
function.=C2=A0 Hence the negative length reaching memcmp().</div><div><br>=
</div><div>Would it make sense to just have &lt;&gt; follow the same leaf-v=
s-internal<br>pattern as the other strategies; negate equality on a leaf, a=
nd<br>otherwise recurse?=C2=A0 An internal page can always contain some<br>=
non-equal value, so I don&#39;t think we lose anything by descending.<br>I =
put together a small patch along those lines (plus a bit(8) test<br>that go=
es through internal pages); it&#39;s attached if it&#39;s useful.<br>(PS. T=
he test exercises the path but passes on a plain build; the OOB<br>read sho=
ws up under ASan (negative-size-param).)<br><br>I didn&#39;t touch the fixe=
d-size path in btree_utils_num.c, since its<br>internal keys look like they=
&#39;re exact lower/upper values, but<br>please correct me if I&#39;m missi=
ng something there.<br><br>Regards,<br>Ayush<br></div></div></div>

--00000000000078a050065477efab--

--00000000000078a052065477efad
Content-Type: application/octet-stream;
	name="0001-Fix-btree_gist-strategy-on-internal-index-pages.patch"
Content-Disposition: attachment;
	filename="0001-Fix-btree_gist-strategy-on-internal-index-pages.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_mqifa6750>
X-Attachment-Id: f_mqifa6750

RnJvbSBhNTNiM2Y1OWE3ZDAxZjAxZDIxZDdmMTlmZDZkMzRhOWEzODYwMjI4IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBBeXVzaCBUaXdhcmkgPGF5dXNodGl3YXJpLnNsZzAxQGdtYWls
LmNvbT4KRGF0ZTogV2VkLCAxNyBKdW4gMjAyNiAyMjo1MjozNiArMDUzMApTdWJqZWN0OiBbUEFU
Q0ggdjFdIEZpeCBidHJlZV9naXN0IDw+IHN0cmF0ZWd5IG9uIGludGVybmFsIGluZGV4IHBhZ2Vz
CgpnYnRfdmFyX2NvbnNpc3RlbnQoKSBoYW5kbGVkIHRoZSA8PiAoQnRyZWVHaXN0Tm90RXF1YWwp
IHN0cmF0ZWd5IHdpdGhvdXQKZGlzdGluZ3Vpc2hpbmcgbGVhZiBmcm9tIGludGVybmFsIHBhZ2Vz
LCB1bmxpa2UgZXZlcnkgb3RoZXIgc3RyYXRlZ3kuCkludGVybmFsIGtleXMgYXJlIGxvc3N5LCB0
cnVuY2F0ZWQgcHJlZml4ZXMsIGFuZCBmb3IgYml0L3ZhcmJpdCB0aGUKbGVhZi10by1ub2RlIHRy
YW5zZm9ybSBkcm9wcyB0aGUgVmFyQml0IGxlbmd0aCBoZWFkZXIsIHNvIHN1Y2ggYSBrZXkgaXMK
bm90IGEgdmFsaWQgdmFsdWUgb2YgdGhlIGluZGV4ZWQgdHlwZS4KClBhc3NpbmcgaXQgdG8gdGhl
IHR5cGUncyBlcXVhbGl0eSBmdW5jdGlvbiBpcyBhIHR5cGUgY29uZnVzaW9uLiAgRm9yCmJpdC92
YXJiaXQgaXQgcmVhY2hlcyBiaXRfY21wKCksIHdoZXJlIFZBUkJJVEJZVEVTKCkgY29tcHV0ZXMg
VkFSU0laRSAtIDg7CmEga2V5IHdpdGggVkFSU0laRSA8IDggeWllbGRzIGEgbmVnYXRpdmUgbGVu
Z3RoIHRoYXQsIHBhc3NlZCB0byBtZW1jbXAoKQphcyBzaXplX3QsIGJlY29tZXMgaHVnZSBhbmQg
cmVhZHMgb3V0IG9mIGJvdW5kcyAoZXZlbnR1YWxseSBjcmFzaGluZyB0aGUKYmFja2VuZCkuICBF
dmVuIHdpdGhvdXQgYSBjcmFzaCBpdCBjYW4gZ2l2ZSB3cm9uZyBlcXVhbGl0eSByZXN1bHRzIGFu
ZApkZWZlYXQgYW4gZXhjbHVzaW9uIGNvbnN0cmFpbnQgZGVjbGFyZWQgd2l0aCA8Pi4KCk1ha2Ug
dGhlIDw+IGJyYW5jaCBkaXN0aW5ndWlzaCBsZWFmIGFuZCBpbnRlcm5hbCBwYWdlczogb24gYSBs
ZWFmLCBuZWdhdGUKdGhlIGVxdWFsaXR5IHRlc3QgYWdhaW5zdCB0aGUgc3RvcmVkIHZhbHVlOyBv
biBhbiBpbnRlcm5hbCBwYWdlLCByZWN1cnNlCnVuY29uZGl0aW9uYWxseS4gIFRoZSBmaXggaXMg
aW4gdGhlIHNoYXJlZCBnYnRfdmFyX2NvbnNpc3RlbnQoKSBhbmQgc28KY292ZXJzIGV2ZXJ5IHZh
cmlhYmxlLWxlbmd0aCB0eXBlLiAgQWRkIGEgcmVncmVzc2lvbiB0ZXN0LgotLS0KIGNvbnRyaWIv
YnRyZWVfZ2lzdC9idHJlZV91dGlsc192YXIuYyAgICAgIHwgMTQgKysrKysrKysrKystLQogY29u
dHJpYi9idHJlZV9naXN0L2V4cGVjdGVkL25vdF9lcXVhbC5vdXQgfCAyNCArKysrKysrKysrKysr
KysrKysrKysrKwogY29udHJpYi9idHJlZV9naXN0L3NxbC9ub3RfZXF1YWwuc3FsICAgICAgfCAx
OCArKysrKysrKysrKysrKysrKwogMyBmaWxlcyBjaGFuZ2VkLCA1NCBpbnNlcnRpb25zKCspLCAy
IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2NvbnRyaWIvYnRyZWVfZ2lzdC9idHJlZV91dGls
c192YXIuYyBiL2NvbnRyaWIvYnRyZWVfZ2lzdC9idHJlZV91dGlsc192YXIuYwppbmRleCAyNWMz
YmJlOGVhYy4uNjA0M2RkODExYmMgMTAwNjQ0Ci0tLSBhL2NvbnRyaWIvYnRyZWVfZ2lzdC9idHJl
ZV91dGlsc192YXIuYworKysgYi9jb250cmliL2J0cmVlX2dpc3QvYnRyZWVfdXRpbHNfdmFyLmMK
QEAgLTYxMSw4ICs2MTEsMTggQEAgZ2J0X3Zhcl9jb25zaXN0ZW50KEdCVF9WQVJLRVlfUiAqa2V5
LAogCQkJCQl8fCBnYnRfdmFyX25vZGVfcGZfbWF0Y2goa2V5LCBxdWVyeSwgdGluZm8pOwogCQkJ
YnJlYWs7CiAJCWNhc2UgQnRyZWVHaXN0Tm90RXF1YWxTdHJhdGVneU51bWJlcjoKLQkJCXJldHZh
bCA9ICEodGluZm8tPmZfZXEocXVlcnksIGtleS0+bG93ZXIsIGNvbGxhdGlvbiwgZmxpbmZvKSAm
JgotCQkJCQkgICB0aW5mby0+Zl9lcShxdWVyeSwga2V5LT51cHBlciwgY29sbGF0aW9uLCBmbGlu
Zm8pKTsKKworCQkJLyoKKwkJCSAqIE11c3QgZGlzdGluZ3Vpc2ggbGVhZiBhbmQgaW50ZXJuYWwg
cGFnZXMuICBJbnRlcm5hbCBrZXlzIGFyZQorCQkJICogbG9zc3ksIHRydW5jYXRlZCBwcmVmaXhl
cyB3aG9zZSByZXByZXNlbnRhdGlvbiBtYXkgZGlmZmVyIGZyb20KKwkJCSAqIHRoZSBsZWFmIGZv
cm0gKGUuZy4gYml0L3ZhcmJpdCksIHNvIGZfZXEoKSBvbiB0aGVtIGNhbiBnaXZlCisJCQkgKiB3
cm9uZyBhbnN3ZXJzIG9yIHJlYWQgb3V0IG9mIGJvdW5kcy4gIEFuIGludGVybmFsIHBhZ2UgbWF5
CisJCQkgKiBhbHdheXMgaG9sZCBhIG5vbi1lcXVhbCB2YWx1ZSwgc28ganVzdCByZWN1cnNlIGlu
dG8gaXQuCisJCQkgKi8KKwkJCWlmIChpc19sZWFmKQorCQkJCXJldHZhbCA9ICF0aW5mby0+Zl9l
cShxdWVyeSwga2V5LT5sb3dlciwgY29sbGF0aW9uLCBmbGluZm8pOworCQkJZWxzZQorCQkJCXJl
dHZhbCA9IHRydWU7CiAJCQlicmVhazsKIAkJZGVmYXVsdDoKIAkJCXJldHZhbCA9IGZhbHNlOwpk
aWZmIC0tZ2l0IGEvY29udHJpYi9idHJlZV9naXN0L2V4cGVjdGVkL25vdF9lcXVhbC5vdXQgYi9j
b250cmliL2J0cmVlX2dpc3QvZXhwZWN0ZWQvbm90X2VxdWFsLm91dAppbmRleCA4NWIxZTg2OGE4
Ny4uOTIyOTUwZTBjMmYgMTAwNjQ0Ci0tLSBhL2NvbnRyaWIvYnRyZWVfZ2lzdC9leHBlY3RlZC9u
b3RfZXF1YWwub3V0CisrKyBiL2NvbnRyaWIvYnRyZWVfZ2lzdC9leHBlY3RlZC9ub3RfZXF1YWwu
b3V0CkBAIC0zOSwzICszOSwyNyBAQCBJTlNFUlQgSU5UTyB6b28gVkFMVUVTKDEyMywgJ2xpb24n
KTsKIEVSUk9SOiAgY29uZmxpY3Rpbmcga2V5IHZhbHVlIHZpb2xhdGVzIGV4Y2x1c2lvbiBjb25z
dHJhaW50ICJ6b29fY2FnZV9hbmltYWxfZXhjbCIKIERFVEFJTDogIEtleSAoY2FnZSwgYW5pbWFs
KT0oMTIzLCBsaW9uKSBjb25mbGljdHMgd2l0aCBleGlzdGluZyBrZXkgKGNhZ2UsIGFuaW1hbCk9
KDEyMywgemVicmEpLgogSU5TRVJUIElOVE8gem9vIFZBTFVFUygxMjQsICdsaW9uJyk7CistLSBB
IDw+IHNlYXJjaCBtdXN0IGRlc2NlbmQgaW50ZXJuYWwgcGFnZXMgaW5zdGVhZCBvZiBjb21wYXJp
bmcgdGhlIHF1ZXJ5IHRvIGEKKy0tIHRydW5jYXRlZCBpbnRlcm5hbCBrZXkuICBVc2UgYSBtdWx0
aS1sZXZlbCBpbmRleCBhbmQgY2hlY2sgaXQgYWdyZWVzIHdpdGggc2VxIHNjYW4uCitDUkVBVEUg
VEFCTEUgYml0bmUgKGEgYml0KDgpKTsKK0lOU0VSVCBJTlRPIGJpdG5lIFNFTEVDVCAoZyAlIDI1
Nik6OmJpdCg4KSBGUk9NIGdlbmVyYXRlX3NlcmllcygxLCAyMDAwMCkgZzsKK0NSRUFURSBJTkRF
WCBiaXRuZV9pZHggT04gYml0bmUgVVNJTkcgZ2lzdCAoYSk7CitTRVQgZW5hYmxlX3NlcXNjYW4g
dG8gb247CitTRVQgZW5hYmxlX2luZGV4c2NhbiB0byBvZmY7CitTRVQgZW5hYmxlX2JpdG1hcHNj
YW4gdG8gb2ZmOworU0VMRUNUIGNvdW50KCopIEZST00gYml0bmUgV0hFUkUgYSA8PiBCJzAwMDAw
MDAwJzsKKyBjb3VudCAKKy0tLS0tLS0KKyAxOTkyMgorKDEgcm93KQorCitTRVQgZW5hYmxlX3Nl
cXNjYW4gdG8gb2ZmOworU0VUIGVuYWJsZV9pbmRleHNjYW4gdG8gb247CitTRVQgZW5hYmxlX2Jp
dG1hcHNjYW4gdG8gb247CitTRUxFQ1QgY291bnQoKikgRlJPTSBiaXRuZSBXSEVSRSBhIDw+IEIn
MDAwMDAwMDAnOworIGNvdW50IAorLS0tLS0tLQorIDE5OTIyCisoMSByb3cpCisKK0RST1AgVEFC
TEUgYml0bmU7CmRpZmYgLS1naXQgYS9jb250cmliL2J0cmVlX2dpc3Qvc3FsL25vdF9lcXVhbC5z
cWwgYi9jb250cmliL2J0cmVlX2dpc3Qvc3FsL25vdF9lcXVhbC5zcWwKaW5kZXggNmRmYWM1ZDBh
YWUuLmVjYjEzNWY0N2QyIDEwMDY0NAotLS0gYS9jb250cmliL2J0cmVlX2dpc3Qvc3FsL25vdF9l
cXVhbC5zcWwKKysrIGIvY29udHJpYi9idHJlZV9naXN0L3NxbC9ub3RfZXF1YWwuc3FsCkBAIC0z
NCwzICszNCwyMSBAQCBJTlNFUlQgSU5UTyB6b28gVkFMVUVTKDEyMywgJ3plYnJhJyk7CiBJTlNF
UlQgSU5UTyB6b28gVkFMVUVTKDEyMywgJ3plYnJhJyk7CiBJTlNFUlQgSU5UTyB6b28gVkFMVUVT
KDEyMywgJ2xpb24nKTsKIElOU0VSVCBJTlRPIHpvbyBWQUxVRVMoMTI0LCAnbGlvbicpOworCist
LSBBIDw+IHNlYXJjaCBtdXN0IGRlc2NlbmQgaW50ZXJuYWwgcGFnZXMgaW5zdGVhZCBvZiBjb21w
YXJpbmcgdGhlIHF1ZXJ5IHRvIGEKKy0tIHRydW5jYXRlZCBpbnRlcm5hbCBrZXkuICBVc2UgYSBt
dWx0aS1sZXZlbCBpbmRleCBhbmQgY2hlY2sgaXQgYWdyZWVzIHdpdGggc2VxIHNjYW4uCitDUkVB
VEUgVEFCTEUgYml0bmUgKGEgYml0KDgpKTsKK0lOU0VSVCBJTlRPIGJpdG5lIFNFTEVDVCAoZyAl
IDI1Nik6OmJpdCg4KSBGUk9NIGdlbmVyYXRlX3NlcmllcygxLCAyMDAwMCkgZzsKK0NSRUFURSBJ
TkRFWCBiaXRuZV9pZHggT04gYml0bmUgVVNJTkcgZ2lzdCAoYSk7CisKK1NFVCBlbmFibGVfc2Vx
c2NhbiB0byBvbjsKK1NFVCBlbmFibGVfaW5kZXhzY2FuIHRvIG9mZjsKK1NFVCBlbmFibGVfYml0
bWFwc2NhbiB0byBvZmY7CitTRUxFQ1QgY291bnQoKikgRlJPTSBiaXRuZSBXSEVSRSBhIDw+IEIn
MDAwMDAwMDAnOworCitTRVQgZW5hYmxlX3NlcXNjYW4gdG8gb2ZmOworU0VUIGVuYWJsZV9pbmRl
eHNjYW4gdG8gb247CitTRVQgZW5hYmxlX2JpdG1hcHNjYW4gdG8gb247CitTRUxFQ1QgY291bnQo
KikgRlJPTSBiaXRuZSBXSEVSRSBhIDw+IEInMDAwMDAwMDAnOworCitEUk9QIFRBQkxFIGJpdG5l
OwotLSAKMi4zNC4xCgo=
--00000000000078a052065477efad--