pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix size underestimation of DSA pagemap for odd-sized segments 6+ messages / 1 participants [nested] [flat]
* pgsql: Fix size underestimation of DSA pagemap for odd-sized segments @ 2026-03-09 04:48 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2026-03-09 04:48 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix size underestimation of DSA pagemap for odd-sized segments When make_new_segment() creates an odd-sized segment, the pagemap was only sized based on a number of usable_pages entries, forgetting that a segment also contains metadata pages, and that the FreePageManager uses absolute page indices that cover the entire segment. This miscalculation could cause accesses to pagemap entries to be out of bounds. During subsequent reuse of the allocated segment, allocations landing on pages with indices higher than usable_pages could cause out-of-bounds pagemap reads and/or writes. On write, 'span' pointers are stored into the data area, corrupting the allocated objects. On read (aka during a dsa_free), garbage is interpreted as a span pointer, typically crashing the server in dsa_get_address(). The normal geometric path correctly sizes the pagemap for all pages in the segment. The odd-sized path needs to do the same, but it works forward from usable_pages rather than backward from total_size. This commit fixes the sizing of the odd-sized case by adding pagemap entries for the metadata pages after the initial metadata_bytes calculation, using an integer ceiling division to compute the exact number of additional entries needed in one go, avoiding any iteration in the calculation. An assertion is added in the code path for odd-sized segments, ensuring that the pagemap includes the metadata area, and that the result is appropriately sized. This problem would show up depending on the size requested for the allocation of a DSA segment. The reporter has noticed this issue when a parallel hash join makes a DSA allocation large enough to trigger the odd-sized segment path, but it could happen for anything that does a DSA allocation. A regression test is added to test_dsa, down to v17 where the test module has been introduced. This adds a set of cheap tests to check the problem, the new assertion being useful for this purpose. Sami has proposed a test that took a longer time than what I have done here; the test committed is faster and good enough to check the odd-sized allocation path. Author: Paul Bunn <paul.bunn@icloud.com> Reviewed-by: Sami Imseih <samimseih@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/044401dcabac$fe432490$fac96db0$@icloud.com Backpatch-through: 14 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/4da2afd01f938af35c1a52bbf6bc40baa52462f6 Modified Files -------------- src/backend/utils/mmgr/dsa.c | 29 ++++++++++++++++++ src/test/modules/test_dsa/expected/test_dsa.out | 16 ++++++++++ src/test/modules/test_dsa/sql/test_dsa.sql | 7 +++++ src/test/modules/test_dsa/test_dsa--1.0.sql | 4 +++ src/test/modules/test_dsa/test_dsa.c | 40 +++++++++++++++++++++++++ 5 files changed, 96 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix size underestimation of DSA pagemap for odd-sized segments @ 2026-03-09 04:48 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2026-03-09 04:48 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix size underestimation of DSA pagemap for odd-sized segments When make_new_segment() creates an odd-sized segment, the pagemap was only sized based on a number of usable_pages entries, forgetting that a segment also contains metadata pages, and that the FreePageManager uses absolute page indices that cover the entire segment. This miscalculation could cause accesses to pagemap entries to be out of bounds. During subsequent reuse of the allocated segment, allocations landing on pages with indices higher than usable_pages could cause out-of-bounds pagemap reads and/or writes. On write, 'span' pointers are stored into the data area, corrupting the allocated objects. On read (aka during a dsa_free), garbage is interpreted as a span pointer, typically crashing the server in dsa_get_address(). The normal geometric path correctly sizes the pagemap for all pages in the segment. The odd-sized path needs to do the same, but it works forward from usable_pages rather than backward from total_size. This commit fixes the sizing of the odd-sized case by adding pagemap entries for the metadata pages after the initial metadata_bytes calculation, using an integer ceiling division to compute the exact number of additional entries needed in one go, avoiding any iteration in the calculation. An assertion is added in the code path for odd-sized segments, ensuring that the pagemap includes the metadata area, and that the result is appropriately sized. This problem would show up depending on the size requested for the allocation of a DSA segment. The reporter has noticed this issue when a parallel hash join makes a DSA allocation large enough to trigger the odd-sized segment path, but it could happen for anything that does a DSA allocation. A regression test is added to test_dsa, down to v17 where the test module has been introduced. This adds a set of cheap tests to check the problem, the new assertion being useful for this purpose. Sami has proposed a test that took a longer time than what I have done here; the test committed is faster and good enough to check the odd-sized allocation path. Author: Paul Bunn <paul.bunn@icloud.com> Reviewed-by: Sami Imseih <samimseih@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/044401dcabac$fe432490$fac96db0$@icloud.com Backpatch-through: 14 Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a0f38604d92814dbe711d1ac3fecb0e5d054c162 Modified Files -------------- src/backend/utils/mmgr/dsa.c | 29 +++++++++++++++++ src/test/modules/test_dsa/expected/test_dsa.out | 16 ++++++++++ src/test/modules/test_dsa/sql/test_dsa.sql | 7 +++++ src/test/modules/test_dsa/test_dsa--1.0.sql | 4 +++ src/test/modules/test_dsa/test_dsa.c | 41 +++++++++++++++++++++++++ 5 files changed, 97 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix size underestimation of DSA pagemap for odd-sized segments @ 2026-03-09 04:48 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2026-03-09 04:48 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix size underestimation of DSA pagemap for odd-sized segments When make_new_segment() creates an odd-sized segment, the pagemap was only sized based on a number of usable_pages entries, forgetting that a segment also contains metadata pages, and that the FreePageManager uses absolute page indices that cover the entire segment. This miscalculation could cause accesses to pagemap entries to be out of bounds. During subsequent reuse of the allocated segment, allocations landing on pages with indices higher than usable_pages could cause out-of-bounds pagemap reads and/or writes. On write, 'span' pointers are stored into the data area, corrupting the allocated objects. On read (aka during a dsa_free), garbage is interpreted as a span pointer, typically crashing the server in dsa_get_address(). The normal geometric path correctly sizes the pagemap for all pages in the segment. The odd-sized path needs to do the same, but it works forward from usable_pages rather than backward from total_size. This commit fixes the sizing of the odd-sized case by adding pagemap entries for the metadata pages after the initial metadata_bytes calculation, using an integer ceiling division to compute the exact number of additional entries needed in one go, avoiding any iteration in the calculation. An assertion is added in the code path for odd-sized segments, ensuring that the pagemap includes the metadata area, and that the result is appropriately sized. This problem would show up depending on the size requested for the allocation of a DSA segment. The reporter has noticed this issue when a parallel hash join makes a DSA allocation large enough to trigger the odd-sized segment path, but it could happen for anything that does a DSA allocation. A regression test is added to test_dsa, down to v17 where the test module has been introduced. This adds a set of cheap tests to check the problem, the new assertion being useful for this purpose. Sami has proposed a test that took a longer time than what I have done here; the test committed is faster and good enough to check the odd-sized allocation path. Author: Paul Bunn <paul.bunn@icloud.com> Reviewed-by: Sami Imseih <samimseih@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/044401dcabac$fe432490$fac96db0$@icloud.com Backpatch-through: 14 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/2543b9ea92fe49c0e4ba1d5adfdc1178c977f371 Modified Files -------------- src/backend/utils/mmgr/dsa.c | 29 +++++++++++++++++ src/test/modules/test_dsa/expected/test_dsa.out | 16 ++++++++++ src/test/modules/test_dsa/sql/test_dsa.sql | 7 +++++ src/test/modules/test_dsa/test_dsa--1.0.sql | 4 +++ src/test/modules/test_dsa/test_dsa.c | 41 +++++++++++++++++++++++++ 5 files changed, 97 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix size underestimation of DSA pagemap for odd-sized segments @ 2026-03-09 04:48 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2026-03-09 04:48 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix size underestimation of DSA pagemap for odd-sized segments When make_new_segment() creates an odd-sized segment, the pagemap was only sized based on a number of usable_pages entries, forgetting that a segment also contains metadata pages, and that the FreePageManager uses absolute page indices that cover the entire segment. This miscalculation could cause accesses to pagemap entries to be out of bounds. During subsequent reuse of the allocated segment, allocations landing on pages with indices higher than usable_pages could cause out-of-bounds pagemap reads and/or writes. On write, 'span' pointers are stored into the data area, corrupting the allocated objects. On read (aka during a dsa_free), garbage is interpreted as a span pointer, typically crashing the server in dsa_get_address(). The normal geometric path correctly sizes the pagemap for all pages in the segment. The odd-sized path needs to do the same, but it works forward from usable_pages rather than backward from total_size. This commit fixes the sizing of the odd-sized case by adding pagemap entries for the metadata pages after the initial metadata_bytes calculation, using an integer ceiling division to compute the exact number of additional entries needed in one go, avoiding any iteration in the calculation. An assertion is added in the code path for odd-sized segments, ensuring that the pagemap includes the metadata area, and that the result is appropriately sized. This problem would show up depending on the size requested for the allocation of a DSA segment. The reporter has noticed this issue when a parallel hash join makes a DSA allocation large enough to trigger the odd-sized segment path, but it could happen for anything that does a DSA allocation. A regression test is added to test_dsa, down to v17 where the test module has been introduced. This adds a set of cheap tests to check the problem, the new assertion being useful for this purpose. Sami has proposed a test that took a longer time than what I have done here; the test committed is faster and good enough to check the odd-sized allocation path. Author: Paul Bunn <paul.bunn@icloud.com> Reviewed-by: Sami Imseih <samimseih@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/044401dcabac$fe432490$fac96db0$@icloud.com Backpatch-through: 14 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/0af5e64e9153afb3214140e5bbebf3dd1743d7ba Modified Files -------------- src/backend/utils/mmgr/dsa.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix size underestimation of DSA pagemap for odd-sized segments @ 2026-03-09 04:48 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2026-03-09 04:48 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix size underestimation of DSA pagemap for odd-sized segments When make_new_segment() creates an odd-sized segment, the pagemap was only sized based on a number of usable_pages entries, forgetting that a segment also contains metadata pages, and that the FreePageManager uses absolute page indices that cover the entire segment. This miscalculation could cause accesses to pagemap entries to be out of bounds. During subsequent reuse of the allocated segment, allocations landing on pages with indices higher than usable_pages could cause out-of-bounds pagemap reads and/or writes. On write, 'span' pointers are stored into the data area, corrupting the allocated objects. On read (aka during a dsa_free), garbage is interpreted as a span pointer, typically crashing the server in dsa_get_address(). The normal geometric path correctly sizes the pagemap for all pages in the segment. The odd-sized path needs to do the same, but it works forward from usable_pages rather than backward from total_size. This commit fixes the sizing of the odd-sized case by adding pagemap entries for the metadata pages after the initial metadata_bytes calculation, using an integer ceiling division to compute the exact number of additional entries needed in one go, avoiding any iteration in the calculation. An assertion is added in the code path for odd-sized segments, ensuring that the pagemap includes the metadata area, and that the result is appropriately sized. This problem would show up depending on the size requested for the allocation of a DSA segment. The reporter has noticed this issue when a parallel hash join makes a DSA allocation large enough to trigger the odd-sized segment path, but it could happen for anything that does a DSA allocation. A regression test is added to test_dsa, down to v17 where the test module has been introduced. This adds a set of cheap tests to check the problem, the new assertion being useful for this purpose. Sami has proposed a test that took a longer time than what I have done here; the test committed is faster and good enough to check the odd-sized allocation path. Author: Paul Bunn <paul.bunn@icloud.com> Reviewed-by: Sami Imseih <samimseih@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/044401dcabac$fe432490$fac96db0$@icloud.com Backpatch-through: 14 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/46c93b705644ad0746dfbd6e0b737fd03e41817f Modified Files -------------- src/backend/utils/mmgr/dsa.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix size underestimation of DSA pagemap for odd-sized segments @ 2026-03-09 04:48 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2026-03-09 04:48 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix size underestimation of DSA pagemap for odd-sized segments When make_new_segment() creates an odd-sized segment, the pagemap was only sized based on a number of usable_pages entries, forgetting that a segment also contains metadata pages, and that the FreePageManager uses absolute page indices that cover the entire segment. This miscalculation could cause accesses to pagemap entries to be out of bounds. During subsequent reuse of the allocated segment, allocations landing on pages with indices higher than usable_pages could cause out-of-bounds pagemap reads and/or writes. On write, 'span' pointers are stored into the data area, corrupting the allocated objects. On read (aka during a dsa_free), garbage is interpreted as a span pointer, typically crashing the server in dsa_get_address(). The normal geometric path correctly sizes the pagemap for all pages in the segment. The odd-sized path needs to do the same, but it works forward from usable_pages rather than backward from total_size. This commit fixes the sizing of the odd-sized case by adding pagemap entries for the metadata pages after the initial metadata_bytes calculation, using an integer ceiling division to compute the exact number of additional entries needed in one go, avoiding any iteration in the calculation. An assertion is added in the code path for odd-sized segments, ensuring that the pagemap includes the metadata area, and that the result is appropriately sized. This problem would show up depending on the size requested for the allocation of a DSA segment. The reporter has noticed this issue when a parallel hash join makes a DSA allocation large enough to trigger the odd-sized segment path, but it could happen for anything that does a DSA allocation. A regression test is added to test_dsa, down to v17 where the test module has been introduced. This adds a set of cheap tests to check the problem, the new assertion being useful for this purpose. Sami has proposed a test that took a longer time than what I have done here; the test committed is faster and good enough to check the odd-sized allocation path. Author: Paul Bunn <paul.bunn@icloud.com> Reviewed-by: Sami Imseih <samimseih@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/044401dcabac$fe432490$fac96db0$@icloud.com Backpatch-through: 14 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/eb11d7a9158d3cd31245b14fb2601e227d947db1 Modified Files -------------- src/backend/utils/mmgr/dsa.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2026-03-09 04:48 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-03-09 04:48 pgsql: Fix size underestimation of DSA pagemap for odd-sized segments Michael Paquier <michael@paquier.xyz> 2026-03-09 04:48 pgsql: Fix size underestimation of DSA pagemap for odd-sized segments Michael Paquier <michael@paquier.xyz> 2026-03-09 04:48 pgsql: Fix size underestimation of DSA pagemap for odd-sized segments Michael Paquier <michael@paquier.xyz> 2026-03-09 04:48 pgsql: Fix size underestimation of DSA pagemap for odd-sized segments Michael Paquier <michael@paquier.xyz> 2026-03-09 04:48 pgsql: Fix size underestimation of DSA pagemap for odd-sized segments Michael Paquier <michael@paquier.xyz> 2026-03-09 04:48 pgsql: Fix size underestimation of DSA pagemap for odd-sized segments Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox