agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v8 2/4] Add function for removing arbitrary nodes in binaryheap. 12+ messages / 2 participants [nested] [flat]
* [PATCH v8 2/4] Add function for removing arbitrary nodes in binaryheap. @ 2023-07-20 16:52 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Nathan Bossart @ 2023-07-20 16:52 UTC (permalink / raw) This commit introduces binaryheap_remove_node(), which can be used to remove any node from a binary heap. The implementation is straightforward. The target node is replaced with the last node in the heap, and then we sift as needed to preserve the heap property. This new function is intended for use in a follow-up commit that will improve the performance of pg_restore. Reviewed-by: Tom Lane Discussion: https://postgr.es/m/3612876.1689443232%40sss.pgh.pa.us --- src/common/binaryheap.c | 29 +++++++++++++++++++++++++++++ src/include/lib/binaryheap.h | 3 +++ 2 files changed, 32 insertions(+) diff --git a/src/common/binaryheap.c b/src/common/binaryheap.c index 39a8243a6d..19e095f1fb 100644 --- a/src/common/binaryheap.c +++ b/src/common/binaryheap.c @@ -215,6 +215,35 @@ binaryheap_remove_first(binaryheap *heap) return result; } +/* + * binaryheap_remove_node + * + * Removes the nth (zero based) node from the heap. The caller must ensure + * that there are at least (n + 1) nodes in the heap. O(log n) worst case. + */ +void +binaryheap_remove_node(binaryheap *heap, int n) +{ + int cmp; + + Assert(!binaryheap_empty(heap) && heap->bh_has_heap_property); + Assert(n >= 0 && n < heap->bh_size); + + /* compare last node to the one that is being removed */ + cmp = heap->bh_compare(heap->bh_nodes[--heap->bh_size], + heap->bh_nodes[n], + heap->bh_arg); + + /* remove the last node, placing it in the vacated entry */ + heap->bh_nodes[n] = heap->bh_nodes[heap->bh_size]; + + /* sift as needed to preserve the heap property */ + if (cmp > 0) + sift_up(heap, n); + else if (cmp < 0) + sift_down(heap, n); +} + /* * binaryheap_replace_first * diff --git a/src/include/lib/binaryheap.h b/src/include/lib/binaryheap.h index 3647aeae65..9525dcaec4 100644 --- a/src/include/lib/binaryheap.h +++ b/src/include/lib/binaryheap.h @@ -59,8 +59,11 @@ extern void binaryheap_build(binaryheap *heap); extern void binaryheap_add(binaryheap *heap, bh_node_type d); extern bh_node_type binaryheap_first(binaryheap *heap); extern bh_node_type binaryheap_remove_first(binaryheap *heap); +extern void binaryheap_remove_node(binaryheap *heap, int n); extern void binaryheap_replace_first(binaryheap *heap, bh_node_type d); #define binaryheap_empty(h) ((h)->bh_size == 0) +#define binaryheap_size(h) ((h)->bh_size) +#define binaryheap_get_node(h, n) ((h)->bh_nodes[n]) #endif /* BINARYHEAP_H */ -- 2.25.1 --gBBFr7Ir9EOA20Yy Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v8-0003-Convert-pg_restore-s-ready_list-to-a-priority-que.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v10 4/5] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- src/test/recovery/t/001_stream_rep.pl | 1 + src/test/subscription/t/001_rep_changes.pl | 1 + 5 files changed, 4 insertions(+), 57 deletions(-) 69.4% src/backend/replication/ 23.4% src/backend/utils/activity/ 3.5% src/test/recovery/t/ 3.6% src/test/subscription/t/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index aecc7a127e6..edf5ac65660 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -571,16 +571,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index a7214d0dc6f..9a136e35b48 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1846,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1867,7 +1862,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1973,8 +1967,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1983,15 +1976,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2894,8 +2878,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2985,9 +2967,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2995,7 +2974,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -3006,21 +2984,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 04d21483d93..ae2952cae89 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl index e9ac67813c7..cfa095ff0a8 100644 --- a/src/test/recovery/t/001_stream_rep.pl +++ b/src/test/recovery/t/001_stream_rep.pl @@ -15,6 +15,7 @@ my $node_primary = PostgreSQL::Test::Cluster->new('primary'); $node_primary->init( allows_streaming => 1, auth_extra => [ '--create-role' => 'repl_role' ]); +$node_primary->append_conf('postgresql.conf', "stats_flush_interval = '1s'"); $node_primary->start; my $backup_name = 'my_backup'; diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index 7d41715ed81..29bae5e1121 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -11,6 +11,7 @@ use Test::More; # Initialize publisher node my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); $node_publisher->init(allows_streaming => 'logical'); +$node_publisher->append_conf('postgresql.conf', "stats_flush_interval = '1s'"); $node_publisher->start; # Create subscriber node -- 2.34.1 --NVvBxFuyV/R+1/8/ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v10-0005-Change-RELATION-and-DATABASE-stats-to-anytime-fl.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v11 4/5] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- src/test/recovery/t/001_stream_rep.pl | 1 + src/test/subscription/t/001_rep_changes.pl | 1 + 5 files changed, 4 insertions(+), 57 deletions(-) 69.4% src/backend/replication/ 23.4% src/backend/utils/activity/ 3.5% src/test/recovery/t/ 3.6% src/test/subscription/t/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index aecc7a127e6..edf5ac65660 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -571,16 +571,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index a7214d0dc6f..9a136e35b48 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1846,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1867,7 +1862,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1973,8 +1967,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1983,15 +1976,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2894,8 +2878,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2985,9 +2967,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2995,7 +2974,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -3006,21 +2984,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 04d21483d93..ae2952cae89 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl index e9ac67813c7..cfa095ff0a8 100644 --- a/src/test/recovery/t/001_stream_rep.pl +++ b/src/test/recovery/t/001_stream_rep.pl @@ -15,6 +15,7 @@ my $node_primary = PostgreSQL::Test::Cluster->new('primary'); $node_primary->init( allows_streaming => 1, auth_extra => [ '--create-role' => 'repl_role' ]); +$node_primary->append_conf('postgresql.conf', "stats_flush_interval = '1s'"); $node_primary->start; my $backup_name = 'my_backup'; diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index 7d41715ed81..29bae5e1121 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -11,6 +11,7 @@ use Test::More; # Initialize publisher node my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); $node_publisher->init(allows_streaming => 'logical'); +$node_publisher->append_conf('postgresql.conf', "stats_flush_interval = '1s'"); $node_publisher->start; # Create subscriber node -- 2.34.1 --2MEBAGW8+kohXisi Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v11-0005-Change-RELATION-and-DATABASE-stats-to-anytime-fl.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v1 2/3] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- 3 files changed, 2 insertions(+), 57 deletions(-) 75.3% src/backend/replication/ 24.6% src/backend/utils/activity/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index a41453530a1..266379c780a 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -553,16 +553,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1ab09655a70..c33185bd337 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1826,7 +1822,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1847,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1958,8 +1952,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1968,15 +1961,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2879,8 +2863,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2975,9 +2957,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2985,7 +2964,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -2996,21 +2974,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index bc8c43b96aa..feae2ae5f44 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* -- 2.34.1 --tjEWjIIwfNIHQLgt Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v1-0003-Add-FLUSH_MIXED-support-and-implement-it-for-RELA.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v3 2/3] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- 3 files changed, 2 insertions(+), 57 deletions(-) 75.3% src/backend/replication/ 24.6% src/backend/utils/activity/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index a41453530a1..266379c780a 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -553,16 +553,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1ab09655a70..c33185bd337 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1826,7 +1822,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1847,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1958,8 +1952,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1968,15 +1961,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2879,8 +2863,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2975,9 +2957,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2985,7 +2964,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -2996,21 +2974,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index bc8c43b96aa..feae2ae5f44 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* -- 2.34.1 --lCc1MWfC7i2uqs8g Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0003-Add-FLUSH_MIXED-support-and-implement-it-for-RELA.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v5 3/4] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- src/test/recovery/t/001_stream_rep.pl | 1 + src/test/subscription/t/001_rep_changes.pl | 1 + 5 files changed, 4 insertions(+), 57 deletions(-) 69.9% src/backend/replication/ 22.8% src/backend/utils/activity/ 3.5% src/test/recovery/t/ 3.6% src/test/subscription/t/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index 24d7ef795cb..1c5ffcab3e0 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -572,16 +572,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index a0e6a3d200c..74102def9c7 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1825,7 +1821,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1846,7 +1841,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1957,8 +1951,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1967,15 +1960,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2878,8 +2862,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2974,9 +2956,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2984,7 +2963,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -2995,21 +2973,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index bc8c43b96aa..feae2ae5f44 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl index e9ac67813c7..c058a5f9b1f 100644 --- a/src/test/recovery/t/001_stream_rep.pl +++ b/src/test/recovery/t/001_stream_rep.pl @@ -15,6 +15,7 @@ my $node_primary = PostgreSQL::Test::Cluster->new('primary'); $node_primary->init( allows_streaming => 1, auth_extra => [ '--create-role' => 'repl_role' ]); +$node_primary->append_conf('postgresql.conf', "stats_flush_interval= '1s'"); $node_primary->start; my $backup_name = 'my_backup'; diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index d7e62e4d488..dda872f7074 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -11,6 +11,7 @@ use Test::More; # Initialize publisher node my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); $node_publisher->init(allows_streaming => 'logical'); +$node_publisher->append_conf('postgresql.conf', "stats_flush_interval= '1s'"); $node_publisher->start; # Create subscriber node -- 2.34.1 --WLtTrIUEfQDmERxy Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v5-0004-Add-FLUSH_MIXED-support-and-implement-it-for-RELA.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v7 4/5] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- src/test/recovery/t/001_stream_rep.pl | 1 + src/test/subscription/t/001_rep_changes.pl | 1 + 5 files changed, 4 insertions(+), 57 deletions(-) 69.4% src/backend/replication/ 23.4% src/backend/utils/activity/ 3.5% src/test/recovery/t/ 3.6% src/test/subscription/t/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index 11b7c114d3b..953ba97ed00 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -571,16 +571,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index a7214d0dc6f..9a136e35b48 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1846,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1867,7 +1862,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1973,8 +1967,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1983,15 +1976,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2894,8 +2878,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2985,9 +2967,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2995,7 +2974,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -3006,21 +2984,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 04d21483d93..ae2952cae89 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl index e9ac67813c7..cfa095ff0a8 100644 --- a/src/test/recovery/t/001_stream_rep.pl +++ b/src/test/recovery/t/001_stream_rep.pl @@ -15,6 +15,7 @@ my $node_primary = PostgreSQL::Test::Cluster->new('primary'); $node_primary->init( allows_streaming => 1, auth_extra => [ '--create-role' => 'repl_role' ]); +$node_primary->append_conf('postgresql.conf', "stats_flush_interval = '1s'"); $node_primary->start; my $backup_name = 'my_backup'; diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index 7d41715ed81..29bae5e1121 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -11,6 +11,7 @@ use Test::More; # Initialize publisher node my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); $node_publisher->init(allows_streaming => 'logical'); +$node_publisher->append_conf('postgresql.conf', "stats_flush_interval = '1s'"); $node_publisher->start; # Create subscriber node -- 2.34.1 --C2xzVbxFmVrP7k6O Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v7-0005-Change-RELATION-and-DATABASE-stats-to-anytime-flu.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v2 2/3] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- 3 files changed, 2 insertions(+), 57 deletions(-) 75.3% src/backend/replication/ 24.6% src/backend/utils/activity/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index a41453530a1..266379c780a 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -553,16 +553,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 1ab09655a70..c33185bd337 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1826,7 +1822,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1847,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1958,8 +1952,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1968,15 +1961,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2879,8 +2863,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2975,9 +2957,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2985,7 +2964,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -2996,21 +2974,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index bc8c43b96aa..feae2ae5f44 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* -- 2.34.1 --OFsrIl+bjhifp5hk Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2-0003-Add-FLUSH_MIXED-support-and-implement-it-for-RELA.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v4 3/4] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- src/test/recovery/t/001_stream_rep.pl | 1 + src/test/subscription/t/001_rep_changes.pl | 1 + 5 files changed, 4 insertions(+), 57 deletions(-) 69.9% src/backend/replication/ 22.8% src/backend/utils/activity/ 3.5% src/test/recovery/t/ 3.6% src/test/subscription/t/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index 6970af3f3ff..dcbe3517b46 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -565,16 +565,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index a0e6a3d200c..74102def9c7 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1825,7 +1821,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1846,7 +1841,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1957,8 +1951,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1967,15 +1960,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2878,8 +2862,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2974,9 +2956,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2984,7 +2963,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -2995,21 +2973,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index bc8c43b96aa..feae2ae5f44 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); } /* diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl index e9ac67813c7..c058a5f9b1f 100644 --- a/src/test/recovery/t/001_stream_rep.pl +++ b/src/test/recovery/t/001_stream_rep.pl @@ -15,6 +15,7 @@ my $node_primary = PostgreSQL::Test::Cluster->new('primary'); $node_primary->init( allows_streaming => 1, auth_extra => [ '--create-role' => 'repl_role' ]); +$node_primary->append_conf('postgresql.conf', "stats_flush_interval= '1s'"); $node_primary->start; my $backup_name = 'my_backup'; diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index d7e62e4d488..dda872f7074 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -11,6 +11,7 @@ use Test::More; # Initialize publisher node my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); $node_publisher->init(allows_streaming => 'logical'); +$node_publisher->append_conf('postgresql.conf', "stats_flush_interval= '1s'"); $node_publisher->start; # Create subscriber node -- 2.34.1 --wR/mWGXukst4NraF Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v4-0004-Add-FLUSH_MIXED-support-and-implement-it-for-RELA.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v6 4/5] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- src/test/recovery/t/001_stream_rep.pl | 1 + src/test/subscription/t/001_rep_changes.pl | 1 + 5 files changed, 4 insertions(+), 57 deletions(-) 69.4% src/backend/replication/ 23.4% src/backend/utils/activity/ 3.4% src/test/recovery/t/ 3.5% src/test/subscription/t/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index 11b7c114d3b..953ba97ed00 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -571,16 +571,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index a7214d0dc6f..9a136e35b48 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1846,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1867,7 +1862,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1973,8 +1967,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1983,15 +1976,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2894,8 +2878,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2985,9 +2967,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2995,7 +2974,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -3006,21 +2984,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 04d21483d93..ae2952cae89 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl index e9ac67813c7..c058a5f9b1f 100644 --- a/src/test/recovery/t/001_stream_rep.pl +++ b/src/test/recovery/t/001_stream_rep.pl @@ -15,6 +15,7 @@ my $node_primary = PostgreSQL::Test::Cluster->new('primary'); $node_primary->init( allows_streaming => 1, auth_extra => [ '--create-role' => 'repl_role' ]); +$node_primary->append_conf('postgresql.conf', "stats_flush_interval= '1s'"); $node_primary->start; my $backup_name = 'my_backup'; diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index 7d41715ed81..bceec2adede 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -11,6 +11,7 @@ use Test::More; # Initialize publisher node my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); $node_publisher->init(allows_streaming => 'logical'); +$node_publisher->append_conf('postgresql.conf', "stats_flush_interval= '1s'"); $node_publisher->start; # Create subscriber node -- 2.34.1 --DxqFthphbM+ha9Dy Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6-0005-Change-RELATION-and-DATABASE-stats-to-anytime-flu.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v8 4/5] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- src/test/recovery/t/001_stream_rep.pl | 1 + src/test/subscription/t/001_rep_changes.pl | 1 + 5 files changed, 4 insertions(+), 57 deletions(-) 69.4% src/backend/replication/ 23.4% src/backend/utils/activity/ 3.5% src/test/recovery/t/ 3.6% src/test/subscription/t/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index 11b7c114d3b..953ba97ed00 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -571,16 +571,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index a7214d0dc6f..9a136e35b48 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1846,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1867,7 +1862,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1973,8 +1967,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1983,15 +1976,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2894,8 +2878,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2985,9 +2967,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2995,7 +2974,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -3006,21 +2984,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 04d21483d93..ae2952cae89 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl index e9ac67813c7..cfa095ff0a8 100644 --- a/src/test/recovery/t/001_stream_rep.pl +++ b/src/test/recovery/t/001_stream_rep.pl @@ -15,6 +15,7 @@ my $node_primary = PostgreSQL::Test::Cluster->new('primary'); $node_primary->init( allows_streaming => 1, auth_extra => [ '--create-role' => 'repl_role' ]); +$node_primary->append_conf('postgresql.conf', "stats_flush_interval = '1s'"); $node_primary->start; my $backup_name = 'my_backup'; diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index 7d41715ed81..29bae5e1121 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -11,6 +11,7 @@ use Test::More; # Initialize publisher node my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); $node_publisher->init(allows_streaming => 'logical'); +$node_publisher->append_conf('postgresql.conf', "stats_flush_interval = '1s'"); $node_publisher->start; # Create subscriber node -- 2.34.1 --OI5irqZsxWxBW9nT Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v8-0005-Change-RELATION-and-DATABASE-stats-to-anytime-flu.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
* [PATCH v9 4/5] Remove useless calls to flush some stats @ 2026-01-06 11:06 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 12+ messages in thread From: Bertrand Drouvot @ 2026-01-06 11:06 UTC (permalink / raw) Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- src/test/recovery/t/001_stream_rep.pl | 1 + src/test/subscription/t/001_rep_changes.pl | 1 + 5 files changed, 4 insertions(+), 57 deletions(-) 69.4% src/backend/replication/ 23.4% src/backend/utils/activity/ 3.5% src/test/recovery/t/ 3.6% src/test/subscription/t/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index 11b7c114d3b..953ba97ed00 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -571,16 +571,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index a7214d0dc6f..9a136e35b48 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1846,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1867,7 +1862,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1973,8 +1967,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1983,15 +1976,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2894,8 +2878,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2985,9 +2967,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2995,7 +2974,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -3006,21 +2984,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 04d21483d93..ae2952cae89 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl index e9ac67813c7..cfa095ff0a8 100644 --- a/src/test/recovery/t/001_stream_rep.pl +++ b/src/test/recovery/t/001_stream_rep.pl @@ -15,6 +15,7 @@ my $node_primary = PostgreSQL::Test::Cluster->new('primary'); $node_primary->init( allows_streaming => 1, auth_extra => [ '--create-role' => 'repl_role' ]); +$node_primary->append_conf('postgresql.conf', "stats_flush_interval = '1s'"); $node_primary->start; my $backup_name = 'my_backup'; diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index 7d41715ed81..29bae5e1121 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -11,6 +11,7 @@ use Test::More; # Initialize publisher node my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); $node_publisher->init(allows_streaming => 'logical'); +$node_publisher->append_conf('postgresql.conf', "stats_flush_interval = '1s'"); $node_publisher->start; # Create subscriber node -- 2.34.1 --aq/6bi8L6WORnI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v9-0005-Change-RELATION-and-DATABASE-stats-to-anytime-flu.patch" ^ permalink raw reply [nested|flat] 12+ messages in thread
end of thread, other threads:[~2026-01-06 11:06 UTC | newest] Thread overview: 12+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-07-20 16:52 [PATCH v8 2/4] Add function for removing arbitrary nodes in binaryheap. Nathan Bossart <[email protected]> 2026-01-06 11:06 [PATCH v10 4/5] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]> 2026-01-06 11:06 [PATCH v11 4/5] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]> 2026-01-06 11:06 [PATCH v1 2/3] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]> 2026-01-06 11:06 [PATCH v3 2/3] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]> 2026-01-06 11:06 [PATCH v5 3/4] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]> 2026-01-06 11:06 [PATCH v7 4/5] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]> 2026-01-06 11:06 [PATCH v2 2/3] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]> 2026-01-06 11:06 [PATCH v4 3/4] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]> 2026-01-06 11:06 [PATCH v6 4/5] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]> 2026-01-06 11:06 [PATCH v8 4/5] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]> 2026-01-06 11:06 [PATCH v9 4/5] Remove useless calls to flush some stats Bertrand Drouvot <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox