public inbox for [email protected]
help / color / mirror / Atom feedFrom: Zhijie Hou (Fujitsu) <[email protected]>
To: Zhijie Hou (Fujitsu) <[email protected]>
To: PostgreSQL Hackers <[email protected]>
Cc: Amit Kapila <[email protected]>
Subject: RE: Fix stats reporting delays in logical parallel apply worker
Date: Fri, 17 Apr 2026 03:35:28 +0000
Message-ID: <TYRPR01MB141956D9C159D7B867A9F2AEE94202@TYRPR01MB14195.jpnprd01.prod.outlook.com> (raw)
In-Reply-To: <TYRPR01MB1419579F217CC4332B615589594202@TYRPR01MB14195.jpnprd01.prod.outlook.com>
References: <TYRPR01MB1419579F217CC4332B615589594202@TYRPR01MB14195.jpnprd01.prod.outlook.com>
On Friday, April 17, 2026 11:01 AM Zhijie Hou (Fujitsu) <[email protected]> wrote:
> Hi,
>
> When implementing another feature, I noticed that parallel apply workers
> currently do not report statistics while idle in their main loop. This can cause
> stats from the last processed transaction to be arbitrarily delayed, especially
> when there are long gaps between streamed transactions.
>
> The issue is demonstrated in 0002, where a TAP test fails when attempting to
> collect stats from a parallel apply worker that has no subsequent transaction
> to
> trigger a stats report.
>
> 0001 fixes this issue by forcing a stats report when the worker is idle in the
> main loop, matching the behavior already present in LogicalRepApplyLoop()
> for
> regular logical apply workers.
Regarding 0002, I realized that the streaming option is now set to 'parallel' by
default so can avoid adjusting the option again. The test needs to be adjusted
to increase the worker limit so that a parallel worker can start. Here are the
updated patches.
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v2-0001-Fix-stats-reporting-delays-in-parallel-apply-work.patch (1.4K, 2-v2-0001-Fix-stats-reporting-delays-in-parallel-apply-work.patch)
download | inline diff:
From d6df6255fdc49e26e65637826358de2a7ce37bc1 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <[email protected]>
Date: Wed, 15 Apr 2026 17:30:03 +0800
Subject: [PATCH v2 1/2] Fix stats reporting delays in parallel apply worker
Parallel apply workers currently do not report statistics while idle in their
main loop. This can cause stats from the last processed transaction to be
arbitrarily delayed, especially when there are long gaps between streamed
transactions.
Fix this by forcing a stats report when the worker is idle in the main loop,
ensuring timely visibility of accumulated statistics.
---
src/backend/replication/logical/applyparallelworker.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/src/backend/replication/logical/applyparallelworker.c b/src/backend/replication/logical/applyparallelworker.c
index 798e8d85b3e..aaa21a5a94a 100644
--- a/src/backend/replication/logical/applyparallelworker.c
+++ b/src/backend/replication/logical/applyparallelworker.c
@@ -815,6 +815,15 @@ LogicalParallelApplyLoop(shm_mq_handle *mqh)
if (rc & WL_LATCH_SET)
ResetLatch(MyLatch);
+
+ /*
+ * Force stats reporting to avoid long delays. There can be long
+ * idle gaps before the leader assigns the next transaction, and
+ * the only opportunity to report stats during such gaps is
+ * here.
+ */
+ if ((rc & WL_TIMEOUT) && !IsTransactionState())
+ pgstat_report_stat(true);
}
}
else
--
2.53.0.windows.2
[application/octet-stream] v2-0002-Test-the-stats-report-in-parallel-apply-worker.patch (2.0K, 3-v2-0002-Test-the-stats-report-in-parallel-apply-worker.patch)
download | inline diff:
From bc74d2584adbb601aa6a0fd5f023190f7bfaad27 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <[email protected]>
Date: Wed, 15 Apr 2026 16:59:53 +0800
Subject: [PATCH v2 2/2] Test the stats report in parallel apply worker
---
src/test/subscription/t/026_stats.pl | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..cfd8d9b9d00 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,9 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+
+# Increase the worker limit to allow a parallel apply worker for this test.
+$node_subscriber->append_conf('postgresql.conf', "max_logical_replication_workers = 5");
$node_subscriber->start;
@@ -136,6 +139,13 @@ sub create_sub_pub_w_errors
# Truncate test table so that apply worker can continue.
$node_subscriber->safe_psql($db, qq(TRUNCATE $table_name));
+ # Force the publisher to stream the changes to the subscriber immediately so
+ # that the delete_missing conflict can be tested in the parallel apply
+ # worker.
+ $node_publisher->append_conf('postgresql.conf',
+ "debug_logical_replication_streaming = immediate");
+ $node_publisher->reload;
+
# Delete data from the test table on the publisher. This delete operation
# should be skipped on the subscriber since the table is already empty.
$node_publisher->safe_psql($db, qq(DELETE FROM $table_name;));
@@ -151,6 +161,12 @@ sub create_sub_pub_w_errors
or die
qq(Timed out while waiting for delete_missing conflict for subscription '$sub_name');
+ # Reset debug_logical_replication_streaming to allow subsequent tests to
+ # verify non-streaming behavior.
+ $node_publisher->append_conf('postgresql.conf',
+ "debug_logical_replication_streaming = buffered");
+ $node_publisher->reload;
+
return ($pub_name, $sub_name);
}
--
2.53.0.windows.2
view thread (7+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: RE: Fix stats reporting delays in logical parallel apply worker
In-Reply-To: <TYRPR01MB141956D9C159D7B867A9F2AEE94202@TYRPR01MB14195.jpnprd01.prod.outlook.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox