agora inbox for pgsql-hackers@postgresql.orghelp / color / mirror / Atom feed
[PATCH 26/32] exist 9+ messages / 2 participants [nested] [flat]
* [PATCH 26/32] exist @ 2021-03-14 01:36 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 9+ messages in thread From: Justin Pryzby @ 2021-03-14 01:36 UTC (permalink / raw) 5f8727f5a679452f7bbdd6966a1586934dcaa84f --- src/backend/commands/analyze.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 5bdaceefd5..182a133033 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -617,7 +617,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params, * * We assume that VACUUM hasn't set pg_class.reltuples already, even * during a VACUUM ANALYZE. Although VACUUM often updates pg_class, - * exceptions exists. A "VACUUM (ANALYZE, INDEX_CLEANUP OFF)" command + * exceptions exist. A "VACUUM (ANALYZE, INDEX_CLEANUP OFF)" command * will never update pg_class entries for index relations. It's also * possible that an individual index's pg_class entry won't be updated * during VACUUM if the index AM returns NULL from its amvacuumcleanup() -- 2.17.0 --4eRLI4hEmsdu6Npr Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0027-as.patch" ^ permalink raw reply [nested|flat] 9+ messages in thread
* [PATCH v2.8 27/38] aio: Experimental heuristics to increase batching in read_stream.c @ 2025-03-14 16:16 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 9+ messages in thread From: Andres Freund @ 2025-03-14 16:16 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- src/backend/storage/aio/read_stream.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c index b44cc358f29..0667efe31ca 100644 --- a/src/backend/storage/aio/read_stream.c +++ b/src/backend/storage/aio/read_stream.c @@ -422,6 +422,30 @@ read_stream_start_pending_read(ReadStream *stream) static void read_stream_look_ahead(ReadStream *stream) { + /* + * Batch-submitting multiple IOs is more efficient than doing so + * one-by-one. If we just ramp up to the max, we'll only be allowed to + * submit one io_combine_limit sized IO. Defer submitting IO in that case. + * + * FIXME: This needs better heuristics. + */ +#if 1 + if (!stream->sync_mode && stream->distance > (io_combine_limit * 8)) + { + if (stream->pinned_buffers + stream->pending_read_nblocks > ((stream->distance * 3) / 4)) + { +#if 0 + ereport(LOG, + errmsg("reduce reduce reduce: pinned: %d, pending: %d, distance: %d", + stream->pinned_buffers, + stream->pending_read_nblocks, + stream->distance)); +#endif + return; + } + } +#endif + /* * Allow amortizing the cost of submitting IO over multiple IOs. This * requires that we don't do any operations that could lead to a deadlock -- 2.48.1.76.g4e746b1a31.dirty --ow5flh3n247znjrs Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2.8-0028-aio-Add-test_aio-module.patch" ^ permalink raw reply [nested|flat] 9+ messages in thread
* [PATCH v2.9 19/30] aio: Experimental heuristics to increase batching in read_stream.c @ 2025-03-15 16:29 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 9+ messages in thread From: Andres Freund @ 2025-03-15 16:29 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- src/backend/storage/aio/read_stream.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c index d7b395b86a3..887be2b3961 100644 --- a/src/backend/storage/aio/read_stream.c +++ b/src/backend/storage/aio/read_stream.c @@ -418,6 +418,30 @@ read_stream_start_pending_read(ReadStream *stream) static void read_stream_look_ahead(ReadStream *stream) { + /* + * Batch-submitting multiple IOs is more efficient than doing so + * one-by-one. If we just ramp up to the max, we'll only be allowed to + * submit one io_combine_limit sized IO. Defer submitting IO in that case. + * + * FIXME: This needs better heuristics. + */ +#if 1 + if (!stream->sync_mode && stream->distance > (io_combine_limit * 8)) + { + if (stream->pinned_buffers + stream->pending_read_nblocks > ((stream->distance * 3) / 4)) + { +#if 0 + ereport(LOG, + errmsg("reduce reduce reduce: pinned: %d, pending: %d, distance: %d", + stream->pinned_buffers, + stream->pending_read_nblocks, + stream->distance)); +#endif + return; + } + } +#endif + /* * Allow amortizing the cost of submitting IO over multiple IOs. This * requires that we don't do any operations that could lead to a deadlock -- 2.48.1.76.g4e746b1a31.dirty --23jbdfobqrqxnmx5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2.9-0020-aio-Add-test_aio-module.patch" ^ permalink raw reply [nested|flat] 9+ messages in thread
* [PATCH v2.12 18/28] aio: Experimental heuristics to increase batching in read_stream.c @ 2025-03-18 18:40 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 9+ messages in thread From: Andres Freund @ 2025-03-18 18:40 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- src/backend/storage/aio/read_stream.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c index f675168e89a..039d7dc71a5 100644 --- a/src/backend/storage/aio/read_stream.c +++ b/src/backend/storage/aio/read_stream.c @@ -403,6 +403,30 @@ read_stream_start_pending_read(ReadStream *stream) static void read_stream_look_ahead(ReadStream *stream) { + /* + * Batch-submitting multiple IOs is more efficient than doing so + * one-by-one. If we just ramp up to the max, we'll only be allowed to + * submit one io_combine_limit sized IO. Defer submitting IO in that case. + * + * FIXME: This needs better heuristics. + */ +#if 1 + if (!stream->sync_mode && stream->distance > (io_combine_limit * 8)) + { + if (stream->pinned_buffers + stream->pending_read_nblocks > ((stream->distance * 3) / 4)) + { +#if 0 + ereport(LOG, + errmsg("reduce reduce reduce: pinned: %d, pending: %d, distance: %d", + stream->pinned_buffers, + stream->pending_read_nblocks, + stream->distance)); +#endif + return; + } + } +#endif + /* * Allow amortizing the cost of submitting IO over multiple IOs. This * requires that we don't do any operations that could lead to a deadlock -- 2.48.1.76.g4e746b1a31.dirty --5i73spx2p4vwf7fe Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2.12-0019-aio-Implement-smgr-md-fd-write-support.patch" ^ permalink raw reply [nested|flat] 9+ messages in thread
* [PATCH v2.13 18/28] aio: Experimental heuristics to increase batching in read_stream.c @ 2025-03-18 18:40 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 9+ messages in thread From: Andres Freund @ 2025-03-18 18:40 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- src/backend/storage/aio/read_stream.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c index 36c54fb695b..cec93129f58 100644 --- a/src/backend/storage/aio/read_stream.c +++ b/src/backend/storage/aio/read_stream.c @@ -404,6 +404,30 @@ read_stream_start_pending_read(ReadStream *stream) static void read_stream_look_ahead(ReadStream *stream) { + /* + * Batch-submitting multiple IOs is more efficient than doing so + * one-by-one. If we just ramp up to the max, we'll only be allowed to + * submit one io_combine_limit sized IO. Defer submitting IO in that case. + * + * FIXME: This needs better heuristics. + */ +#if 1 + if (!stream->sync_mode && stream->distance > (io_combine_limit * 8)) + { + if (stream->pinned_buffers + stream->pending_read_nblocks > ((stream->distance * 3) / 4)) + { +#if 0 + ereport(LOG, + errmsg("reduce reduce reduce: pinned: %d, pending: %d, distance: %d", + stream->pinned_buffers, + stream->pending_read_nblocks, + stream->distance)); +#endif + return; + } + } +#endif + /* * Allow amortizing the cost of submitting IO over multiple IOs. This * requires that we don't do any operations that could lead to a deadlock -- 2.48.1.76.g4e746b1a31.dirty --pro7bqageygxfsvg Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2.13-0019-aio-Implement-smgr-md-fd-write-support.patch" ^ permalink raw reply [nested|flat] 9+ messages in thread
* [PATCH v2.14 19/29] aio: Experimental heuristics to increase batching in read_stream.c @ 2025-03-18 18:40 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 9+ messages in thread From: Andres Freund @ 2025-03-18 18:40 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- src/backend/storage/aio/read_stream.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c index 36c54fb695b..cec93129f58 100644 --- a/src/backend/storage/aio/read_stream.c +++ b/src/backend/storage/aio/read_stream.c @@ -404,6 +404,30 @@ read_stream_start_pending_read(ReadStream *stream) static void read_stream_look_ahead(ReadStream *stream) { + /* + * Batch-submitting multiple IOs is more efficient than doing so + * one-by-one. If we just ramp up to the max, we'll only be allowed to + * submit one io_combine_limit sized IO. Defer submitting IO in that case. + * + * FIXME: This needs better heuristics. + */ +#if 1 + if (!stream->sync_mode && stream->distance > (io_combine_limit * 8)) + { + if (stream->pinned_buffers + stream->pending_read_nblocks > ((stream->distance * 3) / 4)) + { +#if 0 + ereport(LOG, + errmsg("reduce reduce reduce: pinned: %d, pending: %d, distance: %d", + stream->pinned_buffers, + stream->pending_read_nblocks, + stream->distance)); +#endif + return; + } + } +#endif + /* * Allow amortizing the cost of submitting IO over multiple IOs. This * requires that we don't do any operations that could lead to a deadlock -- 2.48.1.76.g4e746b1a31.dirty --m2na7lgr3zfazgom Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2.14-0020-aio-Implement-smgr-md-fd-write-support.patch" ^ permalink raw reply [nested|flat] 9+ messages in thread
* [PATCH v2.15 08/18] aio: Experimental heuristics to increase batching in read_stream.c @ 2025-03-18 18:40 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 9+ messages in thread From: Andres Freund @ 2025-03-18 18:40 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- src/backend/storage/aio/read_stream.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c index 36c54fb695b..cec93129f58 100644 --- a/src/backend/storage/aio/read_stream.c +++ b/src/backend/storage/aio/read_stream.c @@ -404,6 +404,30 @@ read_stream_start_pending_read(ReadStream *stream) static void read_stream_look_ahead(ReadStream *stream) { + /* + * Batch-submitting multiple IOs is more efficient than doing so + * one-by-one. If we just ramp up to the max, we'll only be allowed to + * submit one io_combine_limit sized IO. Defer submitting IO in that case. + * + * FIXME: This needs better heuristics. + */ +#if 1 + if (!stream->sync_mode && stream->distance > (io_combine_limit * 8)) + { + if (stream->pinned_buffers + stream->pending_read_nblocks > ((stream->distance * 3) / 4)) + { +#if 0 + ereport(LOG, + errmsg("reduce reduce reduce: pinned: %d, pending: %d, distance: %d", + stream->pinned_buffers, + stream->pending_read_nblocks, + stream->distance)); +#endif + return; + } + } +#endif + /* * Allow amortizing the cost of submitting IO over multiple IOs. This * requires that we don't do any operations that could lead to a deadlock -- 2.48.1.76.g4e746b1a31.dirty --xevce4sdbnyxplun Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2.15-0009-aio-Implement-smgr-md-fd-write-support.patch" ^ permalink raw reply [nested|flat] 9+ messages in thread
* [PATCH v2.11 18/27] aio: Experimental heuristics to increase batching in read_stream.c @ 2025-03-18 18:40 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 9+ messages in thread From: Andres Freund @ 2025-03-18 18:40 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- src/backend/storage/aio/read_stream.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c index 60a841816d8..facf996608e 100644 --- a/src/backend/storage/aio/read_stream.c +++ b/src/backend/storage/aio/read_stream.c @@ -418,6 +418,30 @@ read_stream_start_pending_read(ReadStream *stream) static void read_stream_look_ahead(ReadStream *stream) { + /* + * Batch-submitting multiple IOs is more efficient than doing so + * one-by-one. If we just ramp up to the max, we'll only be allowed to + * submit one io_combine_limit sized IO. Defer submitting IO in that case. + * + * FIXME: This needs better heuristics. + */ +#if 1 + if (!stream->sync_mode && stream->distance > (io_combine_limit * 8)) + { + if (stream->pinned_buffers + stream->pending_read_nblocks > ((stream->distance * 3) / 4)) + { +#if 0 + ereport(LOG, + errmsg("reduce reduce reduce: pinned: %d, pending: %d, distance: %d", + stream->pinned_buffers, + stream->pending_read_nblocks, + stream->distance)); +#endif + return; + } + } +#endif + /* * Allow amortizing the cost of submitting IO over multiple IOs. This * requires that we don't do any operations that could lead to a deadlock -- 2.48.1.76.g4e746b1a31.dirty --bjnmbpad43bpmfxt Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2.11-0019-aio-Implement-smgr-md-fd-write-support.patch" ^ permalink raw reply [nested|flat] 9+ messages in thread
* [PATCH v2.10 16/28] aio: Experimental heuristics to increase batching in read_stream.c @ 2025-03-18 18:40 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 9+ messages in thread From: Andres Freund @ 2025-03-18 18:40 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- src/backend/storage/aio/read_stream.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c index d7b395b86a3..887be2b3961 100644 --- a/src/backend/storage/aio/read_stream.c +++ b/src/backend/storage/aio/read_stream.c @@ -418,6 +418,30 @@ read_stream_start_pending_read(ReadStream *stream) static void read_stream_look_ahead(ReadStream *stream) { + /* + * Batch-submitting multiple IOs is more efficient than doing so + * one-by-one. If we just ramp up to the max, we'll only be allowed to + * submit one io_combine_limit sized IO. Defer submitting IO in that case. + * + * FIXME: This needs better heuristics. + */ +#if 1 + if (!stream->sync_mode && stream->distance > (io_combine_limit * 8)) + { + if (stream->pinned_buffers + stream->pending_read_nblocks > ((stream->distance * 3) / 4)) + { +#if 0 + ereport(LOG, + errmsg("reduce reduce reduce: pinned: %d, pending: %d, distance: %d", + stream->pinned_buffers, + stream->pending_read_nblocks, + stream->distance)); +#endif + return; + } + } +#endif + /* * Allow amortizing the cost of submitting IO over multiple IOs. This * requires that we don't do any operations that could lead to a deadlock -- 2.48.1.76.g4e746b1a31.dirty --w6dfit2y42fwvotd Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2.10-0017-aio-Add-test_aio-module.patch" ^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2025-03-18 18:40 UTC | newest] Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-03-14 01:36 [PATCH 26/32] exist Justin Pryzby <pryzbyj@telsasoft.com> 2025-03-14 16:16 [PATCH v2.8 27/38] aio: Experimental heuristics to increase batching in read_stream.c Andres Freund <andres@anarazel.de> 2025-03-15 16:29 [PATCH v2.9 19/30] aio: Experimental heuristics to increase batching in read_stream.c Andres Freund <andres@anarazel.de> 2025-03-18 18:40 [PATCH v2.12 18/28] aio: Experimental heuristics to increase batching in read_stream.c Andres Freund <andres@anarazel.de> 2025-03-18 18:40 [PATCH v2.13 18/28] aio: Experimental heuristics to increase batching in read_stream.c Andres Freund <andres@anarazel.de> 2025-03-18 18:40 [PATCH v2.14 19/29] aio: Experimental heuristics to increase batching in read_stream.c Andres Freund <andres@anarazel.de> 2025-03-18 18:40 [PATCH v2.15 08/18] aio: Experimental heuristics to increase batching in read_stream.c Andres Freund <andres@anarazel.de> 2025-03-18 18:40 [PATCH v2.11 18/27] aio: Experimental heuristics to increase batching in read_stream.c Andres Freund <andres@anarazel.de> 2025-03-18 18:40 [PATCH v2.10 16/28] aio: Experimental heuristics to increase batching in read_stream.c Andres Freund <andres@anarazel.de>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox