Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vxhA1-00H2uK-0b for pgsql-hackers@arkaria.postgresql.org; Wed, 04 Mar 2026 08:00:49 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1vxh9z-00BMhn-27 for pgsql-hackers@arkaria.postgresql.org; Wed, 04 Mar 2026 08:00:48 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vxh9z-00BMhc-0N for pgsql-hackers@lists.postgresql.org; Wed, 04 Mar 2026 08:00:47 +0000 Received: from ci-2002j-snip4-4.eps.apple.com ([57.103.88.225] helo=outbound.ci.icloud.com) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1vxh9x-00000000L0o-1kLo for pgsql-hackers@postgresql.org; Wed, 04 Mar 2026 08:00:46 +0000 Received: from outbound.ci.icloud.com (unknown [127.0.0.2]) by p00-icloudmta-asmtp-us-central-1k-60-percent-3 (Postfix) with ESMTPS id 1E1D618000EC for ; Wed, 4 Mar 2026 08:00:41 +0000 (UTC) Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=1a1hai; t=1772611244; x=1775203244; bh=GBYaGbsl/Cm17zutLPhQiMTQTroH07BfD5YZFZ2J+Pk=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:x-icloud-hme; b=g+21b3/vnuS6IPFT3UB8MATjnUd26pXmqFQ2Tm/obOjkqx61UIsDrgOkDjp+ovZT+1Ipe0WYOjeToUZ1AdMAAijqEV/MPspGqpQIT4JnBw7kTQ7wvzNO5zaccG7pURvFaZ14vam9FAmdwyUbf/Vs8QU4m/DuzrzEI7s5AMV5TvCJHQPIbtmX0ZncGcd6/gWI3tGrTU4phAhaT+0u9Cbtbi0TZ3YjvOzJnpyX0V6mPmQCWckR1a2iOXLqWFU7EgdOUO9W3ELAcx8ICAWtPwY5p7K0fQe2cUvgioqY3oYztbkVHLTJRqllvSyog8aChyhemI1u8dq5YHCg42XW3TVp5Q== Received: from PAUL (unknown [17.57.156.36]) by p00-icloudmta-asmtp-us-central-1k-60-percent-3 (Postfix) with ESMTPSA id 1BDCC18000B2 for ; Wed, 4 Mar 2026 08:00:41 +0000 (UTC) From: To: Subject: [BUG + PATCH] DSA pagemap out-of-bounds in make_new_segment odd-sized path Date: Wed, 4 Mar 2026 00:00:39 -0800 Message-ID: <044401dcabac$fe432490$fac96db0$@icloud.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0445_01DCAB69.F02080D0" X-Mailer: Microsoft Outlook 16.0 Content-Language: en-us Thread-Index: AQJXa7Y8juQ8VfYvZx4wHnrciGXuhA== X-Authority-Info-Out: v=2.4 cv=W9E1lBWk c=1 sm=1 tr=0 ts=69a7e6aa cx=c_apl:c_pps:t_out a=2G65uMN5HjSv0sBfM2Yj2w==:117 a=2G65uMN5HjSv0sBfM2Yj2w==:17 a=Yq5XynenixoA:10 a=x7bEGLp0ZPQA:10 a=1_K1eIhTUloA:10 a=VkNPw1HP01LnGYTKEx00:22 a=DAwyPP_o2Byb1YXLmDAA:9 a=ePYpYu1eSlWsgg9QHLMA:9 a=CjuIK1q_8ugA:10 a=yMhMjlubAAAA:8 a=SSmOFEACAAAA:8 a=mXnn9AnQOTG12yQfAGMA:9 a=zb61jw-uI1xbmPSt:21 a=gKO2Hq4RSVkA:10 a=UiCQ7L4-1S4A:10 a=hTZeC7Yk6K0A:10 a=frz4AuCg-hUA:10 a=9PsyOtDX7QqekJOmM8YA:9 a=B2y7HmGcmWMA:10 X-Proofpoint-ORIG-GUID: 7IODj68waFFEnCIf5v25oNGqM_QbuJfR X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzA0MDA2MiBTYWx0ZWRfX4LGjUel1h9zB uvHfVgiwHnbQ42t98RKyJmHaskLw7k09EzLiBXd4oHPETCjWFpq+tv5FHO3cqVJc5gfiujgb6jj dD+2YgoAYrcZA+zEN066tC/jcWiswx1nugIldBnXa9e28B51z5rP0h1k3f8l4sbUfWgmOI2BAFZ R75YG0jedkIh1TlScNs7xm8eBmyoereY0ImugghJEKaSGRvSpjAC98z4HzsPwtaK3gthDYxKiya paZL0cWYxO8NP9MiffPLgIL/1DtFBULRnrXzqlbXiim+Y7jBmBV0FUBPRW2zScUuS058JubBLbW 0wfkEZbJOmDc28OaBAMONe9/SJ6z5+A3IiyC5MxT/paDKQUxPJPTEsOOMMPbiY= X-Proofpoint-GUID: 7IODj68waFFEnCIf5v25oNGqM_QbuJfR X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-04_02,2026-03-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxscore=0 adultscore=0 lowpriorityscore=0 phishscore=0 mlxlogscore=803 malwarescore=0 clxscore=1011 spamscore=0 bulkscore=0 classifier=spam authscore=0 adjust=0 reason=mlx scancount=1 engine=8.22.0-2601150000 definitions=main-2603040062 X-JNJ: AAAAAAABteCqIo/SPnh6ARwwYZF4puQoEasdKwhjRORFqhWLSsqBikRk1vsiDPIakbk5sjJqAle/2bBUPLUk9DkhVSzAjaEBGz+HXvSH0jC5hp27FPZntGnfzRogRbiYdl9XSyXPw4nRqrX7F1A0sO1+XEL0IsNea7gO2VmxX8ZxlSSsrxufea6ctrKD72wvKn+Vbq0qkjkf2JbsV3ajQPzsGYZCJ1KI5J4hMrWLBLTPJBAjBNHt23RrAwNEHqOpoPxs9JNrXsLSeFqOBqAbQyv4NDYvlMQ2K1ICTSteLwUTLEILtisOyqjuZku/FzX/e2bDsHqZQEbngMnfiJ/EcFoPa39tbH5SJ/TM6LV5Reg7ZYb4fMz+fNwHjOInKsp3oBr8SwinFSHYuruWV5H4on1U3nJUrbYH9625DQyU3vkBCY6SwBs7d8ZjsQVrL3ivmQrwXeOLeCTUf5KK37duRgh+4WkVjmJWx4Q2c+m4th/tdsYNjL+HS6iz7eQU7gWGvVVJ/TJ4F3BUXSTxaoSLdeoMPNUx3a5RrDMNaogDeOyAskQyAZD8Mk4UCfGDMl/am6ZR3UIaHIZgG3HaBzC/RMmJ7xuMz9DebTK0bIBinX+zJTEijt77/JrjrPSowsnUlgDkm6v8DPPWEZo+bM6uMR8p3IUHI68= List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multipart message in MIME format. ------=_NextPart_000_0445_01DCAB69.F02080D0 Content-Type: multipart/alternative; boundary="----=_NextPart_001_0446_01DCAB69.F02080D0" ------=_NextPart_001_0446_01DCAB69.F02080D0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hi hackers, Sorry for the previously poorly-formatted/threaded email. We've identified a bug in the DSA (Dynamic Shared Memory Area) allocator that causes memory corruption and crashes during parallel hash joins with large data sets. The bug has been present since the DSA implementation was introduced and affects all supported branches. Attached is a minimal fix (5 lines added, 0 changed). == Bug Summary == In make_new_segment() (src/backend/utils/mmgr/dsa.c), there are two paths for computing segment layout: Path 1 (geometric): knows total_size upfront, computes total_pages = total_size / FPM_PAGE_SIZE pagemap entries = total_pages <-- correct Path 2 (odd-sized, when requested > geometric): works forward from usable_pages = requested_pages pagemap entries = usable_pages <-- BUG The pagemap is indexed by absolute page number. The FreePageManager hands out pages with indices from metadata_pages to (metadata_pages + usable_pages - 1). Since metadata_pages >= 1, page indices at the high end of the range exceed usable_pages, making the pagemap accesses out-of-bounds. == How It Was Found on Postgres 15 == Multiple parallel worker backends crashed simultaneously with SIGSEGV in dsa_get_address(), called from dsa_free() in ExecHashTableDetachBatch() during parallel hash join batch cleanup. Stack trace: #0 dsa_get_address (area, dp) at dsa.c:955 #1 dsa_free (area, dp) at dsa.c:839 #2 ExecHashTableDetachBatch (hashtable) at nodeHash.c:3189 #3 ExecParallelHashJoinNewBatch (hjstate) at nodeHashjoin.c:1157 All crashing workers computed the same pageno (196993), which was the last valid FPM page but beyond the pagemap array (usable_pages = 196609). == Root Cause (from core dump analysis) == The crashing segment had: usable_pages = 196,609 (pagemap has this many entries) metadata_pages = 385 total_pages = 196,994 (= metadata_pages + usable_pages) last FPM page = 196,993 (= metadata_pages + usable_pages - 1) pagemap valid indices: 0 .. 196,608 FPM page indices: 385 .. 196,993 Pages 196,609 through 196,993 (385 pages) have no valid pagemap entry. The pagemap array ends 3,072 bytes before the data area starts (padding zone). Most out-of-bounds entries fall in this padding and cause silent corruption. The last 17 entries fall in the data area itself, causing bidirectional corruption: pagemap writes destroy allocated object data, and subsequent pagemap reads return garbage, crashing dsa_free(). == The Fix == After computing metadata_bytes with usable_pages pagemap entries, add entries for the metadata pages themselves: metadata_bytes += ((metadata_bytes / (FPM_PAGE_SIZE - sizeof(dsa_pointer))) + 1) * sizeof(dsa_pointer); The divisor (FPM_PAGE_SIZE - sizeof(dsa_pointer)) = 4088 accounts for the circular dependency: each metadata page costs one pagemap entry (8 bytes), so only 4088 of each 4096-byte metadata page is net-free for other pagemap entries. The +1 absorbs the ceiling. == Repro == It's tricky to create a test that causes a failure, or standalone SQL that will cause the bug to manifest. This is a (long) dormant bug that only causes a crash in rare circumstances. -- Thanks, Paul ------=_NextPart_001_0446_01DCAB69.F02080D0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi = hackers,

 

Sorry for the = previously poorly-formatted/threaded email.

 

We've identified a = bug in the DSA (Dynamic Shared Memory Area) = allocator

that causes memory corruption and crashes = during parallel hash joins with

large data = sets.  The bug has been present since the DSA = implementation

was introduced and affects all supported = branches.

 

Attached is a minimal = fix (5 lines added, 0 changed).

 

=3D=3D Bug Summary = =3D=3D

 

In make_new_segment() = (src/backend/utils/mmgr/dsa.c), there are two = paths

for computing segment = layout:

 

  Path 1 = (geometric): knows total_size upfront, computes

    = total_pages =3D total_size / FPM_PAGE_SIZE

    = pagemap entries =3D = total_pages          &n= bsp;       <-- = correct

 

  Path 2 = (odd-sized, when requested > geometric): works forward = from

    usable_pages =3D = requested_pages

    pagemap entries =3D = usable_pages          &= nbsp;      <-- BUG

 

The pagemap is = indexed by absolute page number.  The FreePageManager = hands

out pages with indices from metadata_pages to = (metadata_pages +

usable_pages - 1).  Since metadata_pages = >=3D 1, page indices at the high

end of the range = exceed usable_pages, making the pagemap accesses

out-of-bounds.

 

=3D=3D How It Was = Found on Postgres 15 =3D=3D

 

Multiple parallel = worker backends crashed simultaneously with SIGSEGV = in

dsa_get_address(), called from dsa_free() in = ExecHashTableDetachBatch()

during parallel hash = join batch cleanup.

 

Stack = trace:

  #0 dsa_get_address (area, dp) at = dsa.c:955

  #1 dsa_free (area, dp) at = dsa.c:839

  #2 ExecHashTableDetachBatch = (hashtable) at nodeHash.c:3189

  #3 = ExecParallelHashJoinNewBatch (hjstate) at = nodeHashjoin.c:1157

 

All crashing workers = computed the same pageno (196993), which was the = last

valid FPM page but beyond the pagemap array = (usable_pages =3D 196609).

 

=3D=3D Root Cause = (from core dump analysis) =3D=3D

 

The crashing segment = had:

  usable_pages   =3D = 196,609   (pagemap has this many = entries)

  metadata_pages =3D = 385

  total_pages    =3D = 196,994   (=3D metadata_pages + = usable_pages)

  last FPM page  =3D = 196,993   (=3D metadata_pages + usable_pages - = 1)

 

  pagemap valid = indices: 0 .. 196,608

  FPM page = indices:      385 .. = 196,993

 

Pages 196,609 through = 196,993 (385 pages) have no valid pagemap entry.

 

The pagemap array = ends 3,072 bytes before the data area starts = (padding

zone).  Most out-of-bounds entries fall = in this padding and cause silent

corruption.  The = last 17 entries fall in the data area itself, = causing

bidirectional corruption: pagemap writes = destroy allocated object data,

and subsequent = pagemap reads return garbage, crashing = dsa_free().

 

=3D=3D The Fix = =3D=3D

 

After computing = metadata_bytes with usable_pages pagemap entries, = add

entries for the metadata pages = themselves:

 

  metadata_bytes = +=3D

      = ((metadata_bytes / (FPM_PAGE_SIZE - sizeof(dsa_pointer))) + 1) = *

      = sizeof(dsa_pointer);

 

The divisor = (FPM_PAGE_SIZE - sizeof(dsa_pointer)) =3D 4088 accounts = for

the circular dependency: each metadata page = costs one pagemap entry

(8 bytes), so only 4088 of each 4096-byte = metadata page is net-free for

other pagemap = entries.  The +1 absorbs the ceiling.

 

 

=3D=3D Repro = =3D=3D

It's tricky to create a test that causes a = failure, or standalone SQL that will cause

the bug to manifest. = This is a (long) dormant bug that only causes a crash in = rare

circumstances.

 

--

Thanks,

Paul

------=_NextPart_001_0446_01DCAB69.F02080D0-- ------=_NextPart_000_0445_01DCAB69.F02080D0 Content-Type: application/octet-stream; name="0001-Fix-DSA-pagemap-undersizing-in-make_new_segment.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0001-Fix-DSA-pagemap-undersizing-in-make_new_segment.patch"