postgres.git / summary / log / commit / refs
commit 06fb5612c970b3af95aca3db5a955669b07537ca
Author: Thomas Munro <tmunro@postgresql.org>
Date: Wed Mar 19 02:23:12 2025 +0000
Increase io_combine_limit range to 1MB.
The default of 128kB is unchanged, but the upper limit is changed from
32 blocks to 128 blocks, unless the operating system's IOV_MAX is too
low. Some other RDBMSes seem to cap their multi-block buffer pool I/O
around this number, and it seems useful to allow experimentation.
The concrete change is to our definition of PG_IOV_MAX, which provides
the maximum for io_combine_limit and io_max_combine_limit. It also
affects a couple of other places that work with arrays of struct iovec
or smaller objects on the stack, so we still don't want to use the
system IOV_MAX directly without a clamp: it is not under our control and
likely to be 1024. 128 seems acceptable for our current usage.
For Windows, we can't use real scatter/gather yet, so we continue to
define our own IOV_MAX value of 16 and emulate preadv()/pwritev() with
loops. Someone would need to research the trade-offs of raising that
number.
NB if trying to see this working: you might temporarily need to hack
BAS_BULKREAD to be bigger, since otherwise the obvious way of "a very
big SELECT" is limited by that for now.
Suggested-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CA%2BhUKG%2B2T9p-%2BzM6Eeou-RAJjTML6eit1qn26f9twznX59qtCA%40mail.gmail.com
doc/src/sgml/config.sgml | 4 ++++
src/backend/storage/aio/read_stream.c | 7 ++++---
src/backend/utils/misc/postgresql.conf.sample | 4 ++--
src/include/port/pg_iovec.h | 8 ++++++--
4 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 2988865b116..4fd377d1c5f 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2638,6 +2638,8 @@ include_dir 'conf.d'
This parameter can only be set in
the <filename>postgresql.conf</filename> file or on the server
command line.
+ The maximum possible size depends on the operating system and block
+ size, but is typically 1MB on Unix and 128kB on Windows.
The default is 128kB.
</para>
</listitem>
@@ -2655,6 +2657,8 @@ include_dir 'conf.d'
higher than the <varname>io_max_combine_limit</varname> parameter, the
lower value will silently be used instead, so both may need to be raised
to increase the I/O size.
+ The maximum possible size depends on the operating system and block
+ size, but is typically 1MB on Unix and 128kB on Windows.
The default is 128kB.
</para>
</listitem>
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index d65fa07b44c..45bdf819d57 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -515,9 +515,10 @@ read_stream_begin_impl(int flags,
* finishes we don't want to have to wait for its buffers to be consumed
* before starting a new one.
*
- * Be careful not to allow int16 to overflow (even though that's not
- * possible with the current GUC range limits), allowing also for the
- * spare entry and the overflow space.
+ * Be careful not to allow int16 to overflow. That is possible with the
+ * current GUC range limits, so this is an artificial limit of ~32k
+ * buffers and we'd need to adjust the types to exceed that. We also have
+ * to allow for the spare entry and the overflow space.
*/
max_pinned_buffers = (max_ios + 1) * io_combine_limit;
max_pinned_buffers = Min(max_pinned_buffers,
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 66bda60f4ca..6abd1baeac8 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -200,9 +200,9 @@
#backend_flush_after = 0 # measured in pages, 0 disables
#effective_io_concurrency = 16 # 1-1000; 0 disables prefetching
#maintenance_io_concurrency = 16 # 1-1000; 0 disables prefetching
-#io_max_combine_limit = 128kB # usually 1-32 blocks (depends on OS)
+#io_max_combine_limit = 128kB # usually 1-128 blocks (depends on OS)
# (change requires restart)
-#io_combine_limit = 128kB # usually 1-32 blocks (depends on OS)
+#io_combine_limit = 128kB # usually 1-128 blocks (depends on OS)
#io_method = worker # worker, sync (change requires restart)
#io_max_concurrency = -1 # Max number of IOs that one process
diff --git a/src/include/port/pg_iovec.h b/src/include/port/pg_iovec.h
index d9891d3805d..df40c7208be 100644
--- a/src/include/port/pg_iovec.h
+++ b/src/include/port/pg_iovec.h
@@ -33,8 +33,12 @@ struct iovec
#endif
-/* Define a reasonable maximum that is safe to use on the stack. */
-#define PG_IOV_MAX Min(IOV_MAX, 32)
+/*
+ * Define a reasonable maximum that is safe to use on the stack in arrays of
+ * struct iovec and other small types. The operating system could limit us to
+ * a number as low as 16, but most systems have 1024.
+ */
+#define PG_IOV_MAX Min(IOV_MAX, 128)
/*
* Like preadv(), but with a prefix to remind us of a side-effect: on Windows
[parent: 10f664684751]