agora inbox for pgsql-committers@postgresql.orghelp / color / mirror / Atom feed
pgsql: Make local buffers pin limit more conservative 2+ messages / 1 participants [nested] [flat]
* pgsql: Make local buffers pin limit more conservative @ 2026-04-21 15:05 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 2+ messages in thread From: Melanie Plageman @ 2026-04-21 15:05 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Make local buffers pin limit more conservative GetLocalPinLimit() and GetAdditionalLocalPinLimit(), currently in use only by the read stream, previously allowed a backend to pin all num_temp_buffers local buffers. This meant that the read stream could use every available local buffer for read-ahead, leaving none for other concurrent pin-holders like other read streams and related buffers like the visibility map buffer needed during on-access pruning. This became more noticeable since b46e1e54d07, which allows on-access pruning to set the visibility map, which meant that some scans also needed to pin a page of the VM. It caused a test in src/test/regress/sql/temp.sql to fail in some cases. Cap the local pin limit to num_temp_buffers / 4, providing some headroom. This doesn't guarantee that all needed pins will be available — for example, a backend can still open more cursors than there are buffers — but it makes it less likely that read-ahead will exhaust the pool. Note that these functions are not limited by definition to use in the read stream; however, this cap should be appropriate in other contexts. Reported-by: Alexander Lakhin <exclusion@gmail.com> Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/97529f5a-ec10-46b1-ab50-4653126c6889%40gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/da6874635db2e8ab95150481c992f03ec6094805 Modified Files -------------- src/backend/storage/buffer/localbuf.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 2+ messages in thread
* pgsql: Make local buffers pin limit more conservative @ 2026-08-05 15:44 Melanie Plageman <melanieplageman@gmail.com> 0 siblings, 0 replies; 2+ messages in thread From: Melanie Plageman @ 2026-08-05 15:44 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Make local buffers pin limit more conservative GetLocalPinLimit() and GetAdditionalLocalPinLimit(), currently in use only by the read stream, previously allowed a backend to pin all num_temp_buffers local buffers. This meant that the read stream could use every available local buffer for read-ahead, leaving none for other concurrent pin-holders. In 18, this was reported as a sequential scan of a temporary table failing with "no empty local buffer available": with effective_io_concurrency >= 64 and default io_combine_limit and temp_buffers, the scan's read stream budget, (effective_io_concurrency + 1) * io_combine_limit, meets or exceeds num_temp_buffers, so look-ahead could pin the entire local buffer pool. Any additional pin request made while returning tuples -- such as fetching TOASTed values from the table's (also temporary) TOAST table -- then found no unpinned buffer to evict. Cap the local pin limit to num_temp_buffers / 4, providing some headroom. This doesn't guarantee that all needed pins will be available -- for example, a backend can still open more cursors than there are buffers -- but it makes it less likely that read-ahead will exhaust the pool. This is a backpatch of commit da6874635db, which was applied to master only during the 19 development cycle, where the issue surfaced as a regression test failure after on-access pruning began setting the visibility map. Reports of the failure above on 18 prompted backpatching it now. No backpatch to 17, even though its read stream also allows a single stream to pin the whole pool in principle. There, sequential scans pass READ_STREAM_SEQUENTIAL, which disables fadvise-based look-ahead, so the look-ahead distance cannot grow past io_combine_limit (at most 32 buffers, versus a minimum temp_buffers of 100) and the reported failure is unreachable. The remaining theoretical paths require adversarial settings and have never been reported. Moreover, 17 predates these functions; the fix would have to be reimplemented in LimitAdditionalLocalPins(), making the change not a simple cherry-pick. Reported-by: Induja Sreekanthan <indujas@google.com> Reported-by: Eduard Stepanov <crtxcz@gmail.com> Reported-by: Feike Steenbergen <feikesteenbergen@gmail.com> Reported-by: Alexander Lakhin <exclusion@gmail.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/97529f5a-ec10-46b1-ab50-4653126c6889%40gmail.com Discussion: https://postgr.es/m/flat/CAFMO8-rYPSJbXsDdWDzDdpNi-fQ%2B6bKvgbXwE%2BR%3DsGko4epq0Q%40mail.gmail.com Backpatch-through: 18 Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/ed8050370b7cf117dfb0de218695b1e003a51f36 Modified Files -------------- src/backend/storage/buffer/localbuf.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2026-08-05 15:44 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-04-21 15:05 pgsql: Make local buffers pin limit more conservative Melanie Plageman <melanieplageman@gmail.com> 2026-08-05 15:44 pgsql: Make local buffers pin limit more conservative Melanie Plageman <melanieplageman@gmail.com>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox