pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Increase BAS_BULKREAD based on effective_io_concurrency
2+ messages / 2 participants
[nested] [flat]
* pgsql: Increase BAS_BULKREAD based on effective_io_concurrency
@ 2025-04-08 06:52 Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 2+ messages in thread
From: Andres Freund @ 2025-04-08 06:52 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Increase BAS_BULKREAD based on effective_io_concurrency
Before, BAS_BULKREAD was always of size 256kB. With the default
io_combine_limit of 16, that only allowed 1-2 IOs to be in flight -
insufficient even on very low latency storage.
We don't just want to increase the size to a much larger hardcoded value, as
very large rings (10s of MBs of of buffers), appear to have negative
performance effects when reading in data that the OS has cached (but not when
actually needing to do IO).
To address this, increase the size of BAS_BULKREAD to allow for
io_combine_limit * effective_io_concurrency buffers getting read in. To
prevent the ring being much larger than useful, limit the increased size with
GetPinLimit().
The formula outlined above keeps the ring size to sizes for which we have not
observed performance regressions, unless very large effective_io_concurrency
values are used together with large shared_buffers setting.
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/lqwghabtu2ak4wknzycufqjm5ijnxhb4k73vzphlt2a3wsemcd@gtftg44kdim6
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah@brqs62irg4dt
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/15f0cb26b530b6725a37391738cfc62d4745c49b
Modified Files
--------------
src/backend/storage/buffer/freelist.c | 46 +++++++++++++++++++++++++++++++++--
1 file changed, 44 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: pgsql: Increase BAS_BULKREAD based on effective_io_concurrency
@ 2025-11-28 15:17 Frédéric Yhuel <frederic.yhuel@dalibo.com>
parent: Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 2+ messages in thread
From: Frédéric Yhuel @ 2025-11-28 15:17 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; pgsql-committers@lists.postgresql.org; +Cc: Christophe Courtois <christophe.courtois@dalibo.com>
Le 08/04/2025 à 08:52, Andres Freund a écrit :
> Increase BAS_BULKREAD based on effective_io_concurrency
>
> Before, BAS_BULKREAD was always of size 256kB. With the default
> io_combine_limit of 16, that only allowed 1-2 IOs to be in flight -
> insufficient even on very low latency storage.
Hi,
I think that backend/storage/buffer/README should be updated to reflect
this change.
Unfortunately, I don't understand the subject well enough to propose a
patch, even though it would probably just involve copying and pasting
your comments into the README.
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2025-11-28 15:17 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-04-08 06:52 pgsql: Increase BAS_BULKREAD based on effective_io_concurrency Andres Freund <andres@anarazel.de>
2025-11-28 15:17 ` Frédéric Yhuel <frederic.yhuel@dalibo.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox