To: pgsql-hackers@lists.postgresql.org
From: Paul Bunn <paul.bunn@icloud.com>
Subject: 
 =?utf-8?B?W0JVRyArIFBBVENIXSBEU0EgcGFnZW1hcCBvdXQtb2YtYm91bmRzIGluIG1h?=
 =?utf-8?B?a2VfbmV3X3NlZ21lbnQgb2RkLXNpemVkIHBhdGg=?=
Date: Wed, 4 Mar 2026 00:36:09 +0000 (GMT)
Message-id: <ba161ba1-9d5f-409d-9826-40d076d4ca2b@me.com>
Content-Type: multipart/alternative;
 boundary=Apple-Webmail-42--714bc7c7-73a5-463b-918f-0a976accbed0
MIME-Version: 1.0
In-Reply-To: <aad6-m36oM3YNmSH@paquier.xyz>
References: <tencent_FED33DE297DEA5EC5E888D3B9F7E50D6EF07@qq.com>
 <CAExHW5tpF_H9qt1Y6+765WBN8FY9sqJXaCswKHU5cybLFTKSLg@mail.gmail.com>
 <B708143C-0677-4AC2-8815-86FD25A723A1@gmail.com>
 <aad6-m36oM3YNmSH@paquier.xyz>
Archived-At: 
 <https://www.postgresql.org/message-id/ba161ba1-9d5f-409d-9826-40d076d4ca2b%40me.com>
Precedence: bulk


--Apple-Webmail-42--714bc7c7-73a5-463b-918f-0a976accbed0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
    charset=utf-8;
    format=flowed

Hi hackers, We've identified a bug in the DSA (Dynamic Shared Memory Area)=
 allocator that causes memory corruption and crashes during parallel hash =
joins with large data sets. The bug has been present since the DSA impleme=
ntation was introduced and affects all supported branches. Attached is a m=
inimal fix (5 lines added, 0 changed). =3D=3D Bug Summary =3D=3D In make_n=
ew_segment() (src/backend/utils/mmgr/dsa.c), there are two paths for compu=
ting segment layout: Path 1 (geometric): knows total_size upfront, compute=
s total_pages =3D total_size / FPM_PAGE_SIZE pagemap entries =3D total_pag=
es <-- correct Path 2 (odd-sized, when requested > geometric): works forwa=
rd from usable_pages =3D requested_pages pagemap entries =3D usable_pages =
<-- BUG The pagemap is indexed by absolute page number. The FreePageManage=
r hands out pages with indices from metadata_pages to (metadata_pages + us=
able_pages - 1). Since metadata_pages >=3D 1, page indices at the high end=
 of the range exceed usable_pages, making the pagemap accesses out-of-boun=
ds. =3D=3D How It Was Found on Postgres 15 =3D=3D Multiple parallel worker=
 backends crashed simultaneously with SIGSEGV in dsa_get_address(), called=
 from dsa_free() in ExecHashTableDetachBatch() during parallel hash join b=
atch cleanup. Stack trace: #0 dsa_get_address (area, dp) at dsa.c:955 #1 d=
sa_free (area, dp) at dsa.c:839 #2 ExecHashTableDetachBatch (hashtable) at=
 nodeHash.c:3189 #3 ExecParallelHashJoinNewBatch (hjstate) at nodeHashjoin=
.c:1157 All crashing workers computed the same pageno (196993), which was =
the last valid FPM page but beyond the pagemap array (usable_pages =3D 196=
609). =3D=3D Root Cause (from core dump analysis) =3D=3D The crashing segm=
ent had: usable_pages =3D 196,609 (pagemap has this many entries) metadata=
_pages =3D 385 total_pages =3D 196,994 (=3D metadata_pages + usable_pages)=
 last FPM page =3D 196,993 (=3D metadata_pages + usable_pages - 1) pagemap=
 valid indices: 0 .. 196,608 FPM page indices: 385 .. 196,993 Pages 196,60=
9 through 196,993 (385 pages) have no valid pagemap entry. The pagemap arr=
ay ends 3,072 bytes before the data area starts (padding zone). Most out-o=
f-bounds entries fall in this padding and cause silent corruption. The las=
t 17 entries fall in the data area itself, causing bidirectional corruptio=
n: pagemap writes destroy allocated object data, and subsequent pagemap re=
ads return garbage, crashing dsa_free(). =3D=3D The Fix =3D=3D After compu=
ting metadata_bytes with usable_pages pagemap entries, add entries for the=
 metadata pages themselves: metadata_bytes +=3D ((metadata_bytes / (FPM_PA=
GE_SIZE - sizeof(dsa_pointer))) + 1) * sizeof(dsa_pointer); The divisor (F=
PM_PAGE_SIZE - sizeof(dsa_pointer)) =3D 4088 accounts for the circular dep=
endency: each metadata page costs one pagemap entry (8 bytes), so only 408=
8 of each 4096-byte metadata page is net-free for other pagemap entries. T=
he +1 absorbs the ceiling. -- Hopefully I've followed the correct format f=
or creating a patch -- this is my first post here. Thanks, Paul
--Apple-Webmail-42--714bc7c7-73a5-463b-918f-0a976accbed0
Content-Type: multipart/mixed;
    type="text/html";
    boundary=Apple-Webmail-64--714bc7c7-73a5-463b-918f-0a976accbed0


--Apple-Webmail-64--714bc7c7-73a5-463b-918f-0a976accbed0
Content-Type: multipart/related;
    type="text/html";
    boundary=Apple-Webmail-86--714bc7c7-73a5-463b-918f-0a976accbed0


--Apple-Webmail-86--714bc7c7-73a5-463b-918f-0a976accbed0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
    charset=utf-8;

<html><body><div><div>Hi hackers,</div></div><div><br></div><div>We've identified a bu=
g in the DSA (Dynamic Shared Memory Area) allocator&nbsp;that causes memor=
y corruption and crashes during parallel hash joins with large data sets.&=
nbsp; The bug has been present since the DSA implementation&nbsp;was intro=
duced and affects all supported branches.</div><div><br></div><div>Attache=
d is a minimal fix (5 lines added, 0 changed).</div><div><br></div><div>=3D=
=3D Bug Summary =3D=3D</div><div><br></div><div>In make_new_segment() (src=
/backend/utils/mmgr/dsa.c), there are two paths for computing segment layo=
ut:</div><div><br></div><div>&nbsp; Path 1 (geometric): knows total_size u=
pfront, computes</div><div>&nbsp;&nbsp;&nbsp; total_pages =3D total_size /=
 FPM_PAGE_SIZE</div><div>&nbsp;&nbsp;&nbsp; pagemap entries =3D total_page=
s&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;-- correct</div><div><br></div><div>&nbs=
p; Path 2 (odd-sized, when requested &gt; geometric): works forward from</=
div><div>&nbsp;&nbsp;&nbsp; usable_pages =3D requested_pages</div><div>&nb=
sp;&nbsp;&nbsp; pagemap entries =3D usable_pages&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt=
;-- BUG</div><div><br></div><div>The pagemap is indexed by absolute page n=
umber.&nbsp; The FreePageManager hands out pages with indices from&nbsp; m=
etadata_pages to (metadata_pages + usable_pages - 1).&nbsp; Since metadata=
_pages &gt;=3D 1, page indices at the high&nbsp;end of the range exceed us=
able_pages, making the pagemap accesses out-of-bounds.</div><div><br></div=
><div>=3D=3D How It Was Found on Postgres 15 =3D=3D</div><div><br></div><d=
iv>Multiple parallel worker backends crashed simultaneously with SIGSEGV i=
n&nbsp;dsa_get_address(), called from&nbsp; dsa_free() in ExecHashTableDet=
achBatch() during parallel hash join batch cleanup.</div><div><br></div><d=
iv>Stack trace:</div><div>&nbsp; #0 dsa_get_address (area, dp) at dsa.c:95=
5</div><div>&nbsp; #1 dsa_free (area, dp) at dsa.c:839</div><div>&nbsp; #2=
 ExecHashTableDetachBatch (hashtable) at nodeHash.c:3189</div><div>&nbsp; =
#3 ExecParallelHashJoinNewBatch (hjstate) at nodeHashjoin.c:1157</div><div=
><br></div><div>All crashing workers computed the same pageno (196993), wh=
ich was the last valid FPM page but beyond the pagemap array (usable_pages=
 =3D 196609).</div><div><br></div><div>=3D=3D Root Cause (from core dump a=
nalysis) =3D=3D</div><div><br></div><div>The crashing segment had:</div><d=
iv>&nbsp; usable_pages&nbsp;&nbsp; =3D 196,609&nbsp;&nbsp; (pagemap has th=
is many entries)</div><div>&nbsp; metadata_pages =3D 385</div><div>&nbsp; =
total_pages&nbsp;&nbsp;&nbsp; =3D 196,994&nbsp;&nbsp; (=3D metadata_pages =
+ usable_pages)</div><div>&nbsp; last FPM page&nbsp; =3D 196,993&nbsp;&nbs=
p; (=3D metadata_pages + usable_pages - 1)</div><div><br></div><div>&nbsp;=
 pagemap valid indices: 0 .. 196,608</div><div>&nbsp; FPM page indices:&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp; 385 .. 196,993</div><div><br></div><div>Pages =
196,609 through 196,993 (385 pages) have no valid pagemap entry.</div><div=
><br></div><div>The pagemap array ends 3,072 bytes before the data area st=
arts (padding&nbsp;zone).&nbsp; Most out-of-bounds entries fall in this pa=
dding and cause silent&nbsp;corruption.&nbsp; The last 17 entries fall in =
the data area itself, causing&nbsp;bidirectional corruption: pagemap write=
s destroy allocated object data,&nbsp;and subsequent pagemap reads return =
garbage,&nbsp; crashing dsa_free().</div><div><br></div><div>=3D=3D The Fi=
x =3D=3D</div><div><br></div><div>After computing metadata_bytes with usab=
le_pages pagemap entries, add entries for the metadata pages themselves:</=
div><div><br></div><div>&nbsp; metadata_bytes +=3D</div><div>&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp; ((metadata_bytes / (FPM_PAGE_SIZE - sizeof(dsa_pointer))=
) + 1) *</div><div>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sizeof(dsa_pointer);</di=
v><div><br></div><div>The divisor (FPM_PAGE_SIZE - sizeof(dsa_pointer)) =3D=
 4088 accounts for&nbsp;the circular dependency: each metadata&nbsp; page =
costs one pagemap entry (8 bytes), so only 4088 of each 4096-byte metadata=
 page is net-free for other&nbsp; pagemap entries.&nbsp; The +1 absorbs th=
e ceiling.</div><div><br></div><div>--</div><div>Hopefully I've followed t=
he correct format for creating a patch -- this is my first post here.</div=
><div style=3D"font-family: &quot;SF Pro Text&quot;, system-ui, sans-serif=
; color: rgba(255, 255, 255, 0.98);">Thanks,</div><div>Paul</div><div><br>=
</div><div><br></div></body></html>
--Apple-Webmail-86--714bc7c7-73a5-463b-918f-0a976accbed0--

--Apple-Webmail-64--714bc7c7-73a5-463b-918f-0a976accbed0
Content-Transfer-Encoding: BASE64
Content-Disposition: attachment;
 filename="0001-Fix-DSA-pagemap-undersizing-in-make_new_segment.patch"
Content-Type: application/octet-stream;
 name="0001-Fix-DSA-pagemap-undersizing-in-make_new_segment.patch"
Content-Id: <6dd469e4-7d25-48a1-be38-f47d35d2945e@icloud.com>

RnJvbTogUGF1bCBCdW5uIDxwYXVsLmJ1bm5AaWNsb3VkLmNvbT4KRGF0ZTogTW9uLCAzIE1h
ciAyMDI2IDE2OjQwOjAwIC0wODAwClN1YmplY3Q6IFtQQVRDSF0gRml4IERTQSBwYWdlbWFw
IHVuZGVyc2l6aW5nIGluIG1ha2VfbmV3X3NlZ21lbnQKCldoZW4gbWFrZV9uZXdfc2VnbWVu
dCBjcmVhdGVzIGFuICJvZGQtc2l6ZWQiIHNlZ21lbnQgKHRoZSBwYXRoIHRha2VuIHdoZW4K
dGhlIHJlcXVlc3RlZCBhbGxvY2F0aW9uIGV4Y2VlZHMgdGhlIGdlb21ldHJpYyBzZWdtZW50
IHNpemUpLCB0aGUgcGFnZW1hcAppcyBzaXplZCBmb3IgdXNhYmxlX3BhZ2VzIGVudHJpZXMu
ICBIb3dldmVyLCB0aGUgRnJlZVBhZ2VNYW5hZ2VyIGhhbmRzIG91dApwYWdlcyB1c2luZyBh
YnNvbHV0ZSBwYWdlIGluZGljZXMgdGhhdCByYW5nZSB1cCB0byBtZXRhZGF0YV9wYWdlcyAr
CnVzYWJsZV9wYWdlcyAtIDEuICBTaW5jZSBtZXRhZGF0YV9wYWdlcyA+PSAxIGFsd2F5cywg
dGhlIGxhc3QKbWV0YWRhdGFfcGFnZXMgcGFnZSBpbmRpY2VzIGV4Y2VlZCB0aGUgcGFnZW1h
cCBhcnJheSBib3VuZHMuCgpUaGUgbm9ybWFsIChnZW9tZXRyaWMpIHBhdGggY29ycmVjdGx5
IHNpemVzIHRoZSBwYWdlbWFwIGZvciB0b3RhbF9wYWdlcwooPSB0b3RhbF9zaXplIC8gRlBN
X1BBR0VfU0laRSksIHdoaWNoIGluY2x1ZGVzIGJvdGggbWV0YWRhdGEgYW5kIHVzYWJsZQpw
YWdlcy4gIFRoZSBvZGQtc2l6ZWQgcGF0aCBzaG91bGQgZG8gdGhlIHNhbWUsIGJ1dCBpdCB3
b3JrcyBmb3J3YXJkIGZyb20KdXNhYmxlX3BhZ2VzIHJhdGhlciB0aGFuIGJhY2t3YXJkIGZy
b20gdG90YWxfc2l6ZSwgY3JlYXRpbmcgYSBjaXJjdWxhcgpkZXBlbmRlbmN5OiBtZXRhZGF0
YV9ieXRlcyBkZXBlbmRzIG9uIHRvdGFsX3BhZ2VzIHdoaWNoIGRlcGVuZHMgb24KbWV0YWRh
dGFfYnl0ZXMuCgpUaGlzIGNhbiBiZSByZXNvbHZlZCBpbiBjbG9zZWQgZm9ybS4gIEFmdGVy
IGNvbXB1dGluZyBtZXRhZGF0YV9ieXRlcyBmb3IKdXNhYmxlX3BhZ2VzIGVudHJpZXMsIGFk
ZCBwYWdlbWFwIGVudHJpZXMgZm9yIHRoZSBtZXRhZGF0YSBwYWdlcwp0aGVtc2VsdmVzLiAg
VGhlIGRpdmlzb3IgKEZQTV9QQUdFX1NJWkUgLSBzaXplb2YoZHNhX3BvaW50ZXIpKSBhY2Nv
dW50cwpmb3IgdGhlIGZhY3QgdGhhdCBlYWNoIG1ldGFkYXRhIHBhZ2UgY29uc3VtZXMgb25l
IHBhZ2VtYXAgZW50cnkgKG9mCnNpemVvZihkc2FfcG9pbnRlcikgYnl0ZXMpLCBsZWF2aW5n
IG9ubHkgKEZQTV9QQUdFX1NJWkUgLQpzaXplb2YoZHNhX3BvaW50ZXIpKSBuZXQgYnl0ZXMg
cGVyIG1ldGFkYXRhIHBhZ2UuICBUaGUgKzEgYWJzb3JicyB0aGUKY2VpbGluZy4gIFRoaXMg
eWllbGRzIHRoZSBleGFjdCBtZXRhZGF0YV9wYWdlcyBpbiBhIHNpbmdsZSBleHByZXNzaW9u
CndpdGggbm8gaXRlcmF0aW9uIG5lZWRlZC4KClRoZSBidWcgaXMgb2JzZXJ2YWJsZSB3aGVu
IGEgcGFyYWxsZWwgaGFzaCBqb2luIChvciBhbnkgb3BlcmF0aW9uKSBtYWtlcyBhCkRTQSBh
bGxvY2F0aW9uIGxhcmdlIGVub3VnaCB0byB0cmlnZ2VyIHRoZSBvZGQtc2l6ZWQgc2VnbWVu
dCBwYXRoLiAgRHVyaW5nCnN1YnNlcXVlbnQgcmV1c2Ugb2YgdGhlIHNlZ21lbnQsIGFsbG9j
YXRpb25zIGxhbmRpbmcgb24gcGFnZXMgd2l0aCBpbmRpY2VzCj49IHVzYWJsZV9wYWdlcyBj
YXVzZSBvdXQtb2YtYm91bmRzIHBhZ2VtYXAgcmVhZHMvd3JpdGVzLiAgT24gd3JpdGUsCnNw
YW4gcG9pbnRlcnMgYXJlIHN0b3JlZCBpbnRvIHRoZSBkYXRhIGFyZWEsIGNvcnJ1cHRpbmcg
YWxsb2NhdGVkIG9iamVjdHMuCk9uIHJlYWQgKGR1cmluZyBkc2FfZnJlZSksIGdhcmJhZ2Ug
aXMgaW50ZXJwcmV0ZWQgYXMgYSBzcGFuIHBvaW50ZXIsCnR5cGljYWxseSBjcmFzaGluZyBp
biBkc2FfZ2V0X2FkZHJlc3Mgd2l0aCAiZHNhX2FyZWEgY291bGQgbm90IGF0dGFjaCB0bwpz
ZWdtZW50IiBvciBTSUdTRUdWLgoKVGhlIGJ1ZyBoYXMgYmVlbiBwcmVzZW50IHNpbmNlIHRo
ZSBEU0EgaW1wbGVtZW50YXRpb24gd2FzIGludHJvZHVjZWQgYW5kCmFmZmVjdHMgYWxsIHN1
cHBvcnRlZCB2ZXJzaW9ucy4KCi0tLQogc3JjL2JhY2tlbmQvdXRpbHMvbW1nci9kc2EuYyB8
IDUgKysrKysKIDEgZmlsZSBjaGFuZ2VkLCA1IGluc2VydGlvbnMoKykKCmRpZmYgLS1naXQg
YS9zcmMvYmFja2VuZC91dGlscy9tbWdyL2RzYS5jIGIvc3JjL2JhY2tlbmQvdXRpbHMvbW1n
ci9kc2EuYwppbmRleCBjZTllZGU0Li43ODk3NTFmIDEwMDY0NAotLS0gYS9zcmMvYmFja2Vu
ZC91dGlscy9tbWdyL2RzYS5jCisrKyBiL3NyYy9iYWNrZW5kL3V0aWxzL21tZ3IvZHNhLmMK
QEAgLTIyMDYsNiArMjIwNiwxMSBAQCBtYWtlX25ld19zZWdtZW50KGRzYV9hcmVhICphcmVh
LCBzaXplX3QgcmVxdWVzdGVkX3BhZ2VzKQogCQkJTUFYQUxJR04oc2l6ZW9mKEZyZWVQYWdl
TWFuYWdlcikpICsKIAkJCXVzYWJsZV9wYWdlcyAqIHNpemVvZihkc2FfcG9pbnRlcik7CiAK
KwkJLyogQWxzbyBjb3ZlciB0aGUgbWV0YWRhdGEgcGFnZXMgaW4gdGhlIHBhZ2VtYXAgKCsx
IGZvciByb3VuZGluZykuICovCisJCW1ldGFkYXRhX2J5dGVzICs9CisJCQkoKG1ldGFkYXRh
X2J5dGVzIC8gKEZQTV9QQUdFX1NJWkUgLSBzaXplb2YoZHNhX3BvaW50ZXIpKSkgKyAxKSAq
CisJCQlzaXplb2YoZHNhX3BvaW50ZXIpOworCiAJCS8qIEFkZCBwYWRkaW5nIHVwIHRvIG5l
eHQgcGFnZSBib3VuZGFyeS4gKi8KIAkJaWYgKG1ldGFkYXRhX2J5dGVzICUgRlBNX1BBR0Vf
U0laRSAhPSAwKQogCQkJbWV0YWRhdGFfYnl0ZXMgKz0gRlBNX1BBR0VfU0laRSAtIChtZXRh
ZGF0YV9ieXRlcyAlIEZQTV9QQUdFX1NJWkUpOwotLSAKMi40My43Cgo=
--Apple-Webmail-64--714bc7c7-73a5-463b-918f-0a976accbed0--

--Apple-Webmail-42--714bc7c7-73a5-463b-918f-0a976accbed0--