public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 02/19] doc: Remove 'synchronized' from --no-sync
42+ messages / 9 participants
[nested] [flat]
* [PATCH 02/19] doc: Remove 'synchronized' from --no-sync
@ 2022-02-17 03:07 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Justin Pryzby @ 2022-02-17 03:07 UTC (permalink / raw)
Since it would be more accurate to say "unsynchronized".
The corresponding change was made for pgupgrade.sgml in commit 410aa248
---
doc/src/sgml/ref/pg_rewind.sgml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml
index e808239aa5b..3231f67845a 100644
--- a/doc/src/sgml/ref/pg_rewind.sgml
+++ b/doc/src/sgml/ref/pg_rewind.sgml
@@ -210,7 +210,7 @@ PostgreSQL documentation
to be written safely to disk. This option causes
<command>pg_rewind</command> to return without waiting, which is
faster, but means that a subsequent operating system crash can leave
- the synchronized data directory corrupt. Generally, this option is
+ the data directory corrupt. Generally, this option is
useful for testing but should not be used on a production
installation.
</para>
--
2.17.1
--8X7/QrJGcKSMr1RN
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="0003-doc-json-log-dc686681e0799b12c40f44f85fc5bfd7fed4e57.patch"
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 05/17] doc: Remove 'synchronized' from --no-sync
@ 2022-02-17 03:07 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Justin Pryzby @ 2022-02-17 03:07 UTC (permalink / raw)
The corresponding change was made for pgupgrade.sgml in commit 410aa248
---
doc/src/sgml/ref/pg_rewind.sgml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml
index 33e6bb64ad6..c11f671855f 100644
--- a/doc/src/sgml/ref/pg_rewind.sgml
+++ b/doc/src/sgml/ref/pg_rewind.sgml
@@ -210,7 +210,7 @@ PostgreSQL documentation
to be written safely to disk. This option causes
<command>pg_rewind</command> to return without waiting, which is
faster, but means that a subsequent operating system crash can leave
- the synchronized data directory corrupt. Generally, this option is
+ the data directory corrupt. Generally, this option is
useful for testing but should not be used on a production
installation.
</para>
--
2.17.1
--juZjCTNxrMaZdGZC
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="0006-doc-json-log-dc686681e0799b12c40f44f85fc5bfd7fed4e57.patch"
^ permalink raw reply [nested|flat] 42+ messages in thread
* [PATCH 01/13] doc: Remove 'synchronized' from --no-sync
@ 2022-02-17 03:07 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Justin Pryzby @ 2022-02-17 03:07 UTC (permalink / raw)
Since it would be more accurate to say "unsynchronized".
The corresponding change was made for pgupgrade.sgml in commit 410aa248
---
doc/src/sgml/ref/pg_rewind.sgml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml
index e808239aa5b..3231f67845a 100644
--- a/doc/src/sgml/ref/pg_rewind.sgml
+++ b/doc/src/sgml/ref/pg_rewind.sgml
@@ -210,7 +210,7 @@ PostgreSQL documentation
to be written safely to disk. This option causes
<command>pg_rewind</command> to return without waiting, which is
faster, but means that a subsequent operating system crash can leave
- the synchronized data directory corrupt. Generally, this option is
+ the data directory corrupt. Generally, this option is
useful for testing but should not be used on a production
installation.
</para>
--
2.17.1
--KlAEzMkarCnErv5Q
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="0002-doc-pg_column_compression-we-say-method-not-algorith.patch"
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-21 05:34 Andres Freund <[email protected]>
0 siblings, 1 reply; 42+ messages in thread
From: Andres Freund @ 2022-02-21 05:34 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Amit Kapila <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Hi,
On 2022-02-21 12:56:46 +0900, Masahiko Sawada wrote:
> > The patch you referenced [1] should just store the stats in the
> > pg_stat_subscription view, not pg_stat_subscription_workers.
> >
> > It *does* make sense to keep stats about the number of table syncs that failed
> > etc. But that should be a counter in pg_stat_subscription, not a row in
> > pg_stat_subscription_workers.
>
> We have discussed using pg_stat_subscription before but concluded it's
> not an appropriate place to store error information because it ends up
> keeping cumulative stats mixed with non-cumulative stats.
Well, as we've amply discussed, the non-cumulative stats shouldn't be in the
pgstat subsystem.
> To take a precedent, we used to store accumulative statistics such as
> spill_txns to pg_stat_replication, but then for the same reason we moved
> those statistics to the new stats view, pg_stat_replication_slot. New
> subscription statistics that we're introducing are cumulative statistics
> whereas pg_stat_subscription is a dynamic statistics view.
I'm happy to have cumulative subscriber stats somewhere in pgstats. But it
shouldn't be split by worker or relation, and it shouldn't contain
non-cumulative error information.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-21 07:09 Amit Kapila <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 42+ messages in thread
From: Amit Kapila @ 2022-02-21 07:09 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Mon, Feb 21, 2022 at 11:04 AM Andres Freund <[email protected]> wrote:
>
> On 2022-02-21 12:56:46 +0900, Masahiko Sawada wrote:
>
> > To take a precedent, we used to store accumulative statistics such as
> > spill_txns to pg_stat_replication, but then for the same reason we moved
> > those statistics to the new stats view, pg_stat_replication_slot. New
> > subscription statistics that we're introducing are cumulative statistics
> > whereas pg_stat_subscription is a dynamic statistics view.
>
> I'm happy to have cumulative subscriber stats somewhere in pgstats. But it
> shouldn't be split by worker or relation, and it shouldn't contain
> non-cumulative error information.
>
Fair enough. Then, how about the following keeping the following information:
* subid (subscription id)
* subname (subscription name)
* sync_error_count/sync_failure_count (number of timed table sync failed)
* apply_error_count/apply_failure_count (number of times apply failed)
* sync_success_count (number of table syncs successfully finished)
* apply_commit_count (number of transactions applied successfully)
* apply_rollback_count (number of transactions explicitly rolled back)
* stats_reset (Time at which these statistics were last reset)
The view name could be pg_stat_subscription_lrep,
pg_stat_logical_replication, or something on those lines.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-21 07:48 Andres Freund <[email protected]>
parent: Amit Kapila <[email protected]>
0 siblings, 1 reply; 42+ messages in thread
From: Andres Freund @ 2022-02-21 07:48 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Hi,
On 2022-02-21 12:39:31 +0530, Amit Kapila wrote:
> Fair enough. Then, how about the following keeping the following information:
Mostly sounds good.
> * subid (subscription id)
> * subname (subscription name)
Coming from catalog, via join, I assume?
> * sync_error_count/sync_failure_count (number of timed table sync failed)
> * apply_error_count/apply_failure_count (number of times apply failed)
Yep.
> * sync_success_count (number of table syncs successfully finished)
This one I'm not quite convinced by. You can't rely on precise counts with
pgstats and we should be able to get a better idea from somewhere more
permanent which relations succeeded? But it also doesn't do much harm, so ...
> * apply_commit_count (number of transactions applied successfully)
> * apply_rollback_count (number of transactions explicitly rolled back)
What does "explicit" mean here?
> * stats_reset (Time at which these statistics were last reset)
>
> The view name could be pg_stat_subscription_lrep,
> pg_stat_logical_replication, or something on those lines.
pg_stat_subscription_stats :)
(I really dislike that we have pg_stat_ stuff that's not actually stats, but
something describing the current state, but that ship has well sailed).
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-21 09:19 Amit Kapila <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 3 replies; 42+ messages in thread
From: Amit Kapila @ 2022-02-21 09:19 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Mon, Feb 21, 2022 at 1:18 PM Andres Freund <[email protected]> wrote:
>
> On 2022-02-21 12:39:31 +0530, Amit Kapila wrote:
> > Fair enough. Then, how about the following keeping the following information:
>
> Mostly sounds good.
>
>
> > * subid (subscription id)
> > * subname (subscription name)
>
> Coming from catalog, via join, I assume?
>
The subname would be from pg_subscription catalog similar to what we
are doing now for pg_stat_subscription_workers.
>
> > * sync_error_count/sync_failure_count (number of timed table sync failed)
> > * apply_error_count/apply_failure_count (number of times apply failed)
>
> Yep.
>
>
> > * sync_success_count (number of table syncs successfully finished)
>
> This one I'm not quite convinced by. You can't rely on precise counts with
> pgstats and we should be able to get a better idea from somewhere more
> permanent which relations succeeded? But it also doesn't do much harm, so ...
>
We can get precise information from pg_subscription_rel (rels that are
in ready/finish_copy state) but OTOH, during refresh some of the rels
would have been dropped or if a user creates/refreshes publication
with copy_data = false, then we won't get information about how many
table syncs succeeded? I have also kept this to make the sync
information look consistent considering we have sync_failure_count.
>
> > * apply_commit_count (number of transactions applied successfully)
> > * apply_rollback_count (number of transactions explicitly rolled back)
>
> What does "explicit" mean here?
>
It is for the Rollback Prepared case and probably for streaming of
in-progress transactions that eventually get rolled back.
>
> > * stats_reset (Time at which these statistics were last reset)
> >
> > The view name could be pg_stat_subscription_lrep,
> > pg_stat_logical_replication, or something on those lines.
>
> pg_stat_subscription_stats :)
>
Having *stat* two times in the name sounds slightly odd to me but let
us see what others think. One more option could be
pg_stat_subscription_replication.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-21 16:07 David G. Johnston <[email protected]>
parent: Amit Kapila <[email protected]>
2 siblings, 1 reply; 42+ messages in thread
From: David G. Johnston @ 2022-02-21 16:07 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Andres Freund <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers
On Mon, Feb 21, 2022 at 2:19 AM Amit Kapila <[email protected]> wrote:
> On Mon, Feb 21, 2022 at 1:18 PM Andres Freund <[email protected]> wrote:
>
> > > The view name could be pg_stat_subscription_lrep,
> > > pg_stat_logical_replication, or something on those lines.
> >
> > pg_stat_subscription_stats :)
> >
>
> Having *stat* two times in the name sounds slightly odd to me but let
> us see what others think. One more option could be
> pg_stat_subscription_replication.
>
>
Agreed.
pg_stat_subscription_activity
We already have pg_stat_activity (which may be an argument against the
suggestion...)
David J.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-21 16:17 Andres Freund <[email protected]>
parent: Amit Kapila <[email protected]>
2 siblings, 1 reply; 42+ messages in thread
From: Andres Freund @ 2022-02-21 16:17 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Hi,
On 2022-02-21 14:49:01 +0530, Amit Kapila wrote:
> On Mon, Feb 21, 2022 at 1:18 PM Andres Freund <[email protected]> wrote:
> > > * stats_reset (Time at which these statistics were last reset)
> > >
> > > The view name could be pg_stat_subscription_lrep,
> > > pg_stat_logical_replication, or something on those lines.
> >
> > pg_stat_subscription_stats :)
> Having *stat* two times in the name sounds slightly odd to me but let
> us see what others think. One more option could be
> pg_stat_subscription_replication.
It was a joke, making light of our bad naming in pg_stat_*, not a serious
suggestion...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-22 03:37 Amit Kapila <[email protected]>
parent: David G. Johnston <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Amit Kapila @ 2022-02-22 03:37 UTC (permalink / raw)
To: David G. Johnston <[email protected]>; +Cc: Andres Freund <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers
On Mon, Feb 21, 2022 at 9:37 PM David G. Johnston
<[email protected]> wrote:
>
> On Mon, Feb 21, 2022 at 2:19 AM Amit Kapila <[email protected]> wrote:
>>
>> On Mon, Feb 21, 2022 at 1:18 PM Andres Freund <[email protected]> wrote:
>>
>> > > The view name could be pg_stat_subscription_lrep,
>> > > pg_stat_logical_replication, or something on those lines.
>> >
>> > pg_stat_subscription_stats :)
>> >
>>
>> Having *stat* two times in the name sounds slightly odd to me but let
>> us see what others think. One more option could be
>> pg_stat_subscription_replication.
>>
>
> Agreed.
>
> pg_stat_subscription_activity
>
> We already have pg_stat_activity (which may be an argument against the suggestion...)
>
I don't know if that can be an argument against it but one can imagine
that we record other subscription changes like (change of
publications, etc.). I personally feel it may be better to add
'_replication' in some way like pg_stat_sub_replication_activity but I
am fine either way.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-22 05:45 Masahiko Sawada <[email protected]>
parent: Amit Kapila <[email protected]>
2 siblings, 5 replies; 42+ messages in thread
From: Masahiko Sawada @ 2022-02-22 05:45 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Mon, Feb 21, 2022 at 6:19 PM Amit Kapila <[email protected]> wrote:
>
> On Mon, Feb 21, 2022 at 1:18 PM Andres Freund <[email protected]> wrote:
> >
> > On 2022-02-21 12:39:31 +0530, Amit Kapila wrote:
> > > Fair enough. Then, how about the following keeping the following information:
> >
> > Mostly sounds good.
> >
> >
> > > * subid (subscription id)
> > > * subname (subscription name)
> >
> > Coming from catalog, via join, I assume?
> >
>
> The subname would be from pg_subscription catalog similar to what we
> are doing now for pg_stat_subscription_workers.
I've attached a patch that changes pg_stat_subscription_workers view.
It removes non-cumulative values such as error details such as
error-XID and the error message from the view, and consequently the
view now has only cumulative statistics counters: apply_error_count
and sync_error_count. Since the new view name is under discussion I
temporarily chose pg_stat_subscription_activity.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
Attachments:
[application/octet-stream] v1-0001-Reconsider-pg_stat_subscription_workers-view.patch (59.9K, ../../CAD21AoAtrgBEUU08VvfzRnGS+=9rS=MEq1cTLaZ1OT51xX34dQ@mail.gmail.com/2-v1-0001-Reconsider-pg_stat_subscription_workers-view.patch)
download | inline diff:
From 3ffde62a5a4aa176026ef1df3fb95006750cebed Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <[email protected]>
Date: Tue, 22 Feb 2022 10:21:53 +0900
Subject: [PATCH v1] Reconsider pg_stat_subscription_workers view.
As the result of the discussion, we've concluded that the stats
collector is not an appropriate place to store the error information of
subscription workers.
This commits changes the view so that it stores only statistics
counters: apply_error_count and sync_error_count. The
removed error information such as error-XID and the error message
would be stored in another way in the future in which is more reliable
and persistent.
Removing these error details, since we don't need to record the
error information for apply workers and tablesync workers separately,
the view now has one entry per subscription.
Also, it changes the view name to pg_stat_subscription_activity
since the word "worker" is an implementation detail that we use one
worker for one tablesync.
Discussion: https://postgr.es/m/[email protected]
---
doc/src/sgml/logical-replication.sgml | 2 -
doc/src/sgml/monitoring.sgml | 127 +++-----
src/backend/catalog/system_functions.sql | 6 +-
src/backend/catalog/system_views.sql | 26 +-
src/backend/postmaster/pgstat.c | 308 ++++++++------------
src/backend/replication/logical/worker.c | 44 +--
src/backend/utils/adt/pgstatfuncs.c | 120 ++------
src/include/catalog/pg_proc.dat | 27 +-
src/include/pgstat.h | 99 ++-----
src/test/regress/expected/rules.out | 22 +-
src/test/subscription/t/026_worker_stats.pl | 153 +++-------
src/tools/pgindent/typedefs.list | 5 +-
12 files changed, 287 insertions(+), 652 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 96b4886e08..52b6dbbb37 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -346,8 +346,6 @@
<para>
A conflict will produce an error and will stop the replication; it must be
resolved manually by the user. Details about the conflict can be found in
- <link linkend="monitoring-pg-stat-subscription-workers">
- <structname>pg_stat_subscription_workers</structname></link> and the
subscriber's server log.
</para>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index bf7625d988..fe9500f3f6 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -628,11 +628,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</row>
<row>
- <entry><structname>pg_stat_subscription_workers</structname><indexterm><primary>pg_stat_subscription_workers</primary></indexterm></entry>
- <entry>One row per subscription worker, showing statistics about errors
- that occurred on that subscription worker.
- See <link linkend="monitoring-pg-stat-subscription-workers">
- <structname>pg_stat_subscription_workers</structname></link> for details.
+ <entry><structname>pg_stat_subscription_activity</structname><indexterm><primary>pg_stat_subscription_activity</primary></indexterm></entry>
+ <entry>One row per subscription, showing statistics about subscription
+ activity.
+ See <link linkend="monitoring-pg-stat-subscription-activity">
+ <structname>pg_stat_subscription_activity</structname></link> for details.
</entry>
</row>
@@ -3063,23 +3063,21 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
</sect2>
- <sect2 id="monitoring-pg-stat-subscription-workers">
- <title><structname>pg_stat_subscription_workers</structname></title>
+ <sect2 id="monitoring-pg-stat-subscription-activity">
+ <title><structname>pg_stat_subscription_activity</structname></title>
<indexterm>
- <primary>pg_stat_subscription_workers</primary>
+ <primary>pg_stat_subscription_activity</primary>
</indexterm>
<para>
- The <structname>pg_stat_subscription_workers</structname> view will contain
- one row per subscription worker on which errors have occurred, for workers
- applying logical replication changes and workers handling the initial data
- copy of the subscribed tables. The statistics entry is removed when the
+ The <structname>pg_stat_subscription_activity</structname> view will contain
+ one row per subscription. The statistics entry is removed when the
corresponding subscription is dropped.
</para>
- <table id="pg-stat-subscription-workers" xreflabel="pg_stat_subscription_workers">
- <title><structname>pg_stat_subscription_workers</structname> View</title>
+ <table id="pg-stat-subscription-activity" xreflabel="pg_stat_subscription_activity">
+ <title><structname>pg_stat_subscription_activity</structname> View</title>
<tgroup cols="1">
<thead>
<row>
@@ -3113,69 +3111,19 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>subrelid</structfield> <type>oid</type>
+ <structfield>apply_error_count</structfield> <type>uint8</type>
</para>
<para>
- OID of the relation that the worker is synchronizing; null for the
- main apply worker
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_relid</structfield> <type>oid</type>
- </para>
- <para>
- OID of the relation that the worker was processing when the
- error occurred
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_command</structfield> <type>text</type>
- </para>
- <para>
- Name of command being applied when the error occurred. This field
- is null if the error was reported during the initial data copy.
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_xid</structfield> <type>xid</type>
- </para>
- <para>
- Transaction ID of the publisher node being applied when the error
- occurred. This field is null if the error was reported
- during the initial data copy.
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_count</structfield> <type>uint8</type>
- </para>
- <para>
- Number of consecutive times the error occurred
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_message</structfield> <type>text</type>
- </para>
- <para>
- The error message
+ Number of times the error occurred during the application of changes
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_time</structfield> <type>timestamp with time zone</type>
+ <structfield>sync_error_count</structfield> <type>uint8</type>
</para>
<para>
- Last time at which this error occurred
+ Number of times the error occurred during the initial data copy
</para></entry>
</row>
@@ -5267,6 +5215,24 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_reset_single_subscription_counters</primary>
+ </indexterm>
+ <function>pg_stat_reset_single_subscription_counters</function> ( <type>oid</type> )
+ <returnvalue>void</returnvalue>
+ </para>
+ <para>
+ Resets statistics for a single subscription shown in the
+ <structname>pg_stat_subscription_activity</structname> view to zero.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other users
+ can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -5317,31 +5283,6 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
</para></entry>
</row>
- <row>
- <entry role="func_table_entry"><para role="func_signature">
- <indexterm>
- <primary>pg_stat_reset_subscription_worker</primary>
- </indexterm>
- <function>pg_stat_reset_subscription_worker</function> ( <parameter>subid</parameter> <type>oid</type> <optional>, <parameter>relid</parameter> <type>oid</type> </optional> )
- <returnvalue>void</returnvalue>
- </para>
- <para>
- Resets the statistics of subscription workers running on the
- subscription with <parameter>subid</parameter> shown in the
- <structname>pg_stat_subscription_workers</structname> view. If the
- argument <parameter>relid</parameter> is not <literal>NULL</literal>,
- resets statistics of the subscription worker handling the initial data
- copy of the relation with <parameter>relid</parameter>. Otherwise,
- resets the subscription worker statistics of the main apply worker.
- If the argument <parameter>relid</parameter> is omitted, resets the
- statistics of all subscription workers running on the subscription
- with <parameter>subid</parameter>.
- </para>
- <para>
- This function is restricted to superusers by default, but other users
- can be granted EXECUTE to run the function.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index fd1421788e..50d0661f2f 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -637,11 +637,9 @@ REVOKE EXECUTE ON FUNCTION pg_stat_reset_single_table_counters(oid) FROM public;
REVOKE EXECUTE ON FUNCTION pg_stat_reset_single_function_counters(oid) FROM public;
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_replication_slot(text) FROM public;
-
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_stat_reset_single_subscription_counters(oid) FROM public;
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid, oid) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_stat_reset_replication_slot(text) FROM public;
REVOKE EXECUTE ON FUNCTION lo_import(text) FROM public;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 3cb69b1f87..ffbbe49976 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1264,25 +1264,11 @@ GRANT SELECT (oid, subdbid, subname, subowner, subenabled, subbinary,
substream, subtwophasestate, subslotname, subsynccommit, subpublications)
ON pg_subscription TO public;
-CREATE VIEW pg_stat_subscription_workers AS
+CREATE VIEW pg_stat_subscription_activity AS
SELECT
- w.subid,
+ a.subid,
s.subname,
- w.subrelid,
- w.last_error_relid,
- w.last_error_command,
- w.last_error_xid,
- w.last_error_count,
- w.last_error_message,
- w.last_error_time
- FROM (SELECT
- oid as subid,
- NULL as relid
- FROM pg_subscription
- UNION ALL
- SELECT
- srsubid as subid,
- srrelid as relid
- FROM pg_subscription_rel) sr,
- LATERAL pg_stat_get_subscription_worker(sr.subid, sr.relid) w
- JOIN pg_subscription s ON (w.subid = s.oid);
+ a.apply_error_count,
+ a.sync_error_count
+ FROM pg_subscription as s,
+ LATERAL pg_stat_get_subscription_activity(oid) as a;
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 0646f53098..ec73f22ef3 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -106,7 +106,7 @@
#define PGSTAT_DB_HASH_SIZE 16
#define PGSTAT_TAB_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
-#define PGSTAT_SUBWORKER_HASH_SIZE 32
+#define PGSTAT_SUBSCRIPTION_HASH_SIZE 32
#define PGSTAT_REPLSLOT_HASH_SIZE 32
@@ -322,14 +322,13 @@ NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_no
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
Oid tableoid, bool create);
-static PgStat_StatSubWorkerEntry *pgstat_get_subworker_entry(PgStat_StatDBEntry *dbentry,
- Oid subid, Oid subrelid,
- bool create);
+static PgStat_StatSubEntry *pgstat_get_sub_entry(PgStat_StatDBEntry *dbentry,
+ Oid subid, bool create);
static void pgstat_write_statsfiles(bool permanent, bool allDbs);
static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- HTAB *subworkerhash, bool permanent);
+ HTAB *subhash, bool permanent);
static void backend_read_statsfile(void);
static bool pgstat_write_statsfile_needed(void);
@@ -381,7 +380,7 @@ static void pgstat_recv_disconnect(PgStat_MsgDisconnect *msg, int len);
static void pgstat_recv_replslot(PgStat_MsgReplSlot *msg, int len);
static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
static void pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len);
-static void pgstat_recv_subworker_error(PgStat_MsgSubWorkerError *msg, int len);
+static void pgstat_recv_subscription_error(PgStat_MsgSubscriptionError *msg, int len);
/* ------------------------------------------------------------
* Public functions called from postmaster follow
@@ -1313,13 +1312,13 @@ pgstat_vacuum_stat(void)
}
/*
- * Repeat for subscription workers. Similarly, we needn't bother in the
- * common case where no subscription workers' stats are being collected.
+ * Repeat for subscription. Similarly, we needn't bother in the common
+ * case where no subscription stats are being collected.
*/
- if (dbentry->subworkers != NULL &&
- hash_get_num_entries(dbentry->subworkers) > 0)
+ if (dbentry->subscriptions != NULL &&
+ hash_get_num_entries(dbentry->subscriptions) > 0)
{
- PgStat_StatSubWorkerEntry *subwentry;
+ PgStat_StatSubEntry *subentry;
PgStat_MsgSubscriptionPurge spmsg;
/*
@@ -1331,35 +1330,16 @@ pgstat_vacuum_stat(void)
spmsg.m_databaseid = MyDatabaseId;
spmsg.m_nentries = 0;
- hash_seq_init(&hstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&hstat)) != NULL)
+ hash_seq_init(&hstat, dbentry->subscriptions);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&hstat)) != NULL)
{
- bool exists = false;
- Oid subid = subwentry->key.subid;
+ Oid subid = subentry->subid;
CHECK_FOR_INTERRUPTS();
if (hash_search(htab, (void *) &subid, HASH_FIND, NULL) != NULL)
continue;
- /*
- * It is possible that we have multiple entries for the
- * subscription corresponding to apply worker and tablesync
- * workers. In such cases, we don't need to add the same subid
- * again.
- */
- for (int i = 0; i < spmsg.m_nentries; i++)
- {
- if (spmsg.m_subids[i] == subid)
- {
- exists = true;
- break;
- }
- }
-
- if (exists)
- continue;
-
/* This subscription is dead, add the subid to the message */
spmsg.m_subids[spmsg.m_nentries++] = subid;
@@ -1551,8 +1531,7 @@ pgstat_reset_shared_counters(const char *target)
* ----------
*/
void
-pgstat_reset_single_counter(Oid objoid, Oid subobjoid,
- PgStat_Single_Reset_Type type)
+pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
PgStat_MsgResetsinglecounter msg;
@@ -1563,7 +1542,6 @@ pgstat_reset_single_counter(Oid objoid, Oid subobjoid,
msg.m_databaseid = MyDatabaseId;
msg.m_resettype = type;
msg.m_objectid = objoid;
- msg.m_subobjectid = subobjoid;
pgstat_send(&msg, sizeof(msg));
}
@@ -1949,31 +1927,21 @@ pgstat_report_replslot_drop(const char *slotname)
}
/* ----------
- * pgstat_report_subworker_error() -
+ * pgstat_report_subscription_error() -
*
- * Tell the collector about the subscription worker error.
+ * Tell the collector about the subscription error.
* ----------
*/
void
-pgstat_report_subworker_error(Oid subid, Oid subrelid, Oid relid,
- LogicalRepMsgType command, TransactionId xid,
- const char *errmsg)
+pgstat_report_subscription_error(Oid subid, bool is_apply_error)
{
- PgStat_MsgSubWorkerError msg;
- int len;
+ PgStat_MsgSubscriptionError msg;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBWORKERERROR);
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBSCRIPTIONERROR);
msg.m_databaseid = MyDatabaseId;
msg.m_subid = subid;
- msg.m_subrelid = subrelid;
- msg.m_relid = relid;
- msg.m_command = command;
- msg.m_xid = xid;
- msg.m_timestamp = GetCurrentTimestamp();
- strlcpy(msg.m_message, errmsg, PGSTAT_SUBWORKERERROR_MSGLEN);
-
- len = offsetof(PgStat_MsgSubWorkerError, m_message) + strlen(msg.m_message) + 1;
- pgstat_send(&msg, len);
+ msg.m_is_apply_error = is_apply_error;
+ pgstat_send(&msg, sizeof(PgStat_MsgSubscriptionError));
}
/* ----------
@@ -3000,32 +2968,29 @@ pgstat_fetch_stat_funcentry(Oid func_id)
/*
* ---------
- * pgstat_fetch_stat_subworker_entry() -
+ * pgstat_fetch_stat_subentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for subscription worker or NULL.
+ * the collected statistics for subscription or NULL.
* ---------
*/
-PgStat_StatSubWorkerEntry *
-pgstat_fetch_stat_subworker_entry(Oid subid, Oid subrelid)
+PgStat_StatSubEntry *
+pgstat_fetch_stat_subentry(Oid subid)
{
PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *wentry = NULL;
+ PgStat_StatSubEntry *entry = NULL;
/* Load the stats file if needed */
backend_read_statsfile();
/*
- * Lookup our database, then find the requested subscription worker stats.
+ * Lookup our database, then find the requested subscription stats.
*/
dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->subworkers != NULL)
- {
- wentry = pgstat_get_subworker_entry(dbentry, subid, subrelid,
- false);
- }
+ if (dbentry != NULL && dbentry->subscriptions != NULL)
+ entry = pgstat_get_sub_entry(dbentry, subid, false);
- return wentry;
+ return entry;
}
/*
@@ -3738,12 +3703,12 @@ PgstatCollectorMain(int argc, char *argv[])
pgstat_recv_disconnect(&msg.msg_disconnect, len);
break;
- case PGSTAT_MTYPE_SUBSCRIPTIONPURGE:
- pgstat_recv_subscription_purge(&msg.msg_subscriptionpurge, len);
+ case PGSTAT_MTYPE_SUBSCRIPTIONERROR:
+ pgstat_recv_subscription_error(&msg.msg_subscriptionerror, len);
break;
- case PGSTAT_MTYPE_SUBWORKERERROR:
- pgstat_recv_subworker_error(&msg.msg_subworkererror, len);
+ case PGSTAT_MTYPE_SUBSCRIPTIONPURGE:
+ pgstat_recv_subscription_purge(&msg.msg_subscriptionpurge, len);
break;
default:
@@ -3791,8 +3756,7 @@ PgstatCollectorMain(int argc, char *argv[])
/*
* Subroutine to clear stats in a database entry
*
- * Tables, functions, and subscription workers hashes are initialized
- * to empty.
+ * Tables, functions, and subscriptions hashes are initialized to empty.
*/
static void
reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
@@ -3846,12 +3810,12 @@ reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
&hash_ctl,
HASH_ELEM | HASH_BLOBS);
- hash_ctl.keysize = sizeof(PgStat_StatSubWorkerKey);
- hash_ctl.entrysize = sizeof(PgStat_StatSubWorkerEntry);
- dbentry->subworkers = hash_create("Per-database subscription worker",
- PGSTAT_SUBWORKER_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
+ hash_ctl.keysize = sizeof(Oid);
+ hash_ctl.entrysize = sizeof(PgStat_StatSubEntry);
+ dbentry->subscriptions = hash_create("Per-database subscription",
+ PGSTAT_SUBSCRIPTION_HASH_SIZE,
+ &hash_ctl,
+ HASH_ELEM | HASH_BLOBS);
}
/*
@@ -3876,7 +3840,7 @@ pgstat_get_db_entry(Oid databaseid, bool create)
/*
* If not found, initialize the new one. This creates empty hash tables
- * for tables, functions, and subscription workers, too.
+ * for tables, functions, and subscriptions, too.
*/
if (!found)
reset_dbentry_counters(result);
@@ -3935,29 +3899,23 @@ pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
}
/* ----------
- * pgstat_get_subworker_entry
+ * pgstat_get_sub_entry
*
- * Return subscription worker entry with the given subscription OID and
- * relation OID. If subrelid is InvalidOid, it returns an entry of the
- * apply worker otherwise returns an entry of the table sync worker
- * associated with subrelid. If no subscription worker entry exists,
- * initialize it, if the create parameter is true. Else, return NULL.
+ * Return subscription statistics entry with the given subscription OID.
+ * If no subscription entry exists, initialize it, if the create parameter is
+ * true. Else, return NULL.
* ----------
*/
-static PgStat_StatSubWorkerEntry *
-pgstat_get_subworker_entry(PgStat_StatDBEntry *dbentry, Oid subid, Oid subrelid,
- bool create)
+static PgStat_StatSubEntry *
+pgstat_get_sub_entry(PgStat_StatDBEntry *dbentry, Oid subid, bool create)
{
- PgStat_StatSubWorkerEntry *subwentry;
- PgStat_StatSubWorkerKey key;
+ PgStat_StatSubEntry *subentry;
bool found;
HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
- key.subid = subid;
- key.subrelid = subrelid;
- subwentry = (PgStat_StatSubWorkerEntry *) hash_search(dbentry->subworkers,
- (void *) &key,
- action, &found);
+ subentry = (PgStat_StatSubEntry *) hash_search(dbentry->subscriptions,
+ (void *) &subid,
+ action, &found);
if (!create && !found)
return NULL;
@@ -3965,15 +3923,11 @@ pgstat_get_subworker_entry(PgStat_StatDBEntry *dbentry, Oid subid, Oid subrelid,
/* If not found, initialize the new one */
if (!found)
{
- subwentry->last_error_relid = InvalidOid;
- subwentry->last_error_command = 0;
- subwentry->last_error_xid = InvalidTransactionId;
- subwentry->last_error_count = 0;
- subwentry->last_error_time = 0;
- subwentry->last_error_message[0] = '\0';
+ subentry->apply_error_count = 0;
+ subentry->sync_error_count = 0;
}
- return subwentry;
+ return subentry;
}
/* ----------
@@ -4059,8 +4013,8 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
{
/*
- * Write out the table, function, and subscription-worker stats for
- * this DB into the appropriate per-DB stat file, if required.
+ * Write out the table, function, and subscription stats for this DB
+ * into the appropriate per-DB stat file, if required.
*/
if (allDbs || pgstat_db_requested(dbentry->databaseid))
{
@@ -4177,7 +4131,7 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
HASH_SEQ_STATUS sstat;
PgStat_StatTabEntry *tabentry;
PgStat_StatFuncEntry *funcentry;
- PgStat_StatSubWorkerEntry *subwentry;
+ PgStat_StatSubEntry *subentry;
FILE *fpout;
int32 format_id;
Oid dbid = dbentry->databaseid;
@@ -4233,13 +4187,13 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
}
/*
- * Walk through the database's subscription worker stats table.
+ * Walk through the database's subscription stats table.
*/
- hash_seq_init(&sstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&sstat)) != NULL)
+ hash_seq_init(&sstat, dbentry->subscriptions);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&sstat)) != NULL)
{
fputc('S', fpout);
- rc = fwrite(subwentry, sizeof(PgStat_StatSubWorkerEntry), 1, fpout);
+ rc = fwrite(subentry, sizeof(PgStat_StatSubEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
@@ -4301,8 +4255,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
* files after reading; the in-memory status is now authoritative, and the
* files would be out of date in case somebody else reads them.
*
- * If a 'deep' read is requested, table/function/subscription-worker stats are
- * read, otherwise the table/function/subscription-worker hash tables remain
+ * If a 'deep' read is requested, table/function/subscription stats are
+ * read, otherwise the table/function/subscription hash tables remain
* empty.
* ----------
*/
@@ -4482,7 +4436,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
dbentry->tables = NULL;
dbentry->functions = NULL;
- dbentry->subworkers = NULL;
+ dbentry->subscriptions = NULL;
/*
* In the collector, disregard the timestamp we read from the
@@ -4494,7 +4448,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
dbentry->stats_timestamp = 0;
/*
- * Don't create tables/functions/subworkers hashtables for
+ * Don't create tables/functions/subscriptions hashtables for
* uninteresting databases.
*/
if (onlydb != InvalidOid)
@@ -4520,13 +4474,13 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
&hash_ctl,
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
- hash_ctl.keysize = sizeof(PgStat_StatSubWorkerKey);
- hash_ctl.entrysize = sizeof(PgStat_StatSubWorkerEntry);
+ hash_ctl.keysize = sizeof(Oid);
+ hash_ctl.entrysize = sizeof(PgStat_StatSubEntry);
hash_ctl.hcxt = pgStatLocalContext;
- dbentry->subworkers = hash_create("Per-database subscription worker",
- PGSTAT_SUBWORKER_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ dbentry->subscriptions = hash_create("Per-database subscription",
+ PGSTAT_SUBSCRIPTION_HASH_SIZE,
+ &hash_ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
/*
* If requested, read the data from the database-specific
@@ -4536,7 +4490,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
pgstat_read_db_statsfile(dbentry->databaseid,
dbentry->tables,
dbentry->functions,
- dbentry->subworkers,
+ dbentry->subscriptions,
permanent);
break;
@@ -4615,20 +4569,20 @@ done:
* removed after reading.
*
* Note: this code has the ability to skip storing per-table, per-function, or
- * per-subscription-worker data, if NULL is passed for the corresponding hashtable.
+ * per-subscription data, if NULL is passed for the corresponding hashtable.
* That's not used at the moment though.
* ----------
*/
static void
pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- HTAB *subworkerhash, bool permanent)
+ HTAB *subhash, bool permanent)
{
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
- PgStat_StatSubWorkerEntry subwbuf;
- PgStat_StatSubWorkerEntry *subwentry;
+ PgStat_StatSubEntry subbuf;
+ PgStat_StatSubEntry *subentry;
FILE *fpin;
int32 format_id;
bool found;
@@ -4743,12 +4697,12 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
break;
/*
- * 'S' A PgStat_StatSubWorkerEntry struct describing
- * subscription worker statistics.
+ * 'S' A PgStat_StatSubEntry struct describing subscription
+ * statistics.
*/
case 'S':
- if (fread(&subwbuf, 1, sizeof(PgStat_StatSubWorkerEntry),
- fpin) != sizeof(PgStat_StatSubWorkerEntry))
+ if (fread(&subbuf, 1, sizeof(PgStat_StatSubEntry),
+ fpin) != sizeof(PgStat_StatSubEntry))
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
(errmsg("corrupted statistics file \"%s\"",
@@ -4757,14 +4711,14 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
}
/*
- * Skip if subscription worker data not wanted.
+ * Skip if subscription data not wanted.
*/
- if (subworkerhash == NULL)
+ if (subhash == NULL)
break;
- subwentry = (PgStat_StatSubWorkerEntry *) hash_search(subworkerhash,
- (void *) &subwbuf.key,
- HASH_ENTER, &found);
+ subentry = (PgStat_StatSubEntry *) hash_search(subhash,
+ (void *) &subbuf.subid,
+ HASH_ENTER, &found);
if (found)
{
@@ -4774,7 +4728,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
goto done;
}
- memcpy(subwentry, &subwbuf, sizeof(subwbuf));
+ memcpy(subentry, &subbuf, sizeof(subbuf));
break;
/*
@@ -5450,8 +5404,8 @@ pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
hash_destroy(dbentry->tables);
if (dbentry->functions != NULL)
hash_destroy(dbentry->functions);
- if (dbentry->subworkers != NULL)
- hash_destroy(dbentry->subworkers);
+ if (dbentry->subscriptions != NULL)
+ hash_destroy(dbentry->subscriptions);
if (hash_search(pgStatDBHash,
(void *) &dbid,
@@ -5489,16 +5443,16 @@ pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
hash_destroy(dbentry->tables);
if (dbentry->functions != NULL)
hash_destroy(dbentry->functions);
- if (dbentry->subworkers != NULL)
- hash_destroy(dbentry->subworkers);
+ if (dbentry->subscriptions != NULL)
+ hash_destroy(dbentry->subscriptions);
dbentry->tables = NULL;
dbentry->functions = NULL;
- dbentry->subworkers = NULL;
+ dbentry->subscriptions = NULL;
/*
* Reset database-level stats, too. This creates empty hash tables for
- * tables, functions, and subscription workers.
+ * tables, functions, and subscriptions.
*/
reset_dbentry_counters(dbentry);
}
@@ -5567,14 +5521,9 @@ pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
else if (msg->m_resettype == RESET_FUNCTION)
(void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_SUBWORKER)
- {
- PgStat_StatSubWorkerKey key;
-
- key.subid = msg->m_objectid;
- key.subrelid = msg->m_subobjectid;
- (void) hash_search(dbentry->subworkers, (void *) &key, HASH_REMOVE, NULL);
- }
+ else if (msg->m_resettype == RESET_SUBSCRIPTION)
+ (void) hash_search(dbentry->subscriptions, (void *) &(msg->m_objectid),
+ HASH_REMOVE, NULL);
}
/* ----------
@@ -6126,73 +6075,48 @@ pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
static void
pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len)
{
- HASH_SEQ_STATUS hstat;
PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *subwentry;
dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
/* No need to purge if we don't even know the database */
- if (!dbentry || !dbentry->subworkers)
+ if (!dbentry || !dbentry->subscriptions)
return;
- /* Remove all subscription worker statistics for the given subscriptions */
- hash_seq_init(&hstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&hstat)) != NULL)
+ /*
+ * Process all subscription entries in the message.
+ */
+ for (int i = 0; i < msg->m_nentries; i++)
{
- for (int i = 0; i < msg->m_nentries; i++)
- {
- if (subwentry->key.subid == msg->m_subids[i])
- {
- (void) hash_search(dbentry->subworkers, (void *) &(subwentry->key),
- HASH_REMOVE, NULL);
- break;
- }
- }
+ /* Remove from hashtable if present; we don't care if it's not */
+ (void) hash_search(dbentry->subscriptions,
+ (void *) &(msg->m_subids[i]),
+ HASH_REMOVE, NULL);
}
}
/* ----------
- * pgstat_recv_subworker_error() -
+ * pgstat_recv_subscription_error() -
*
- * Process a SUBWORKERERROR message.
+ * Process a SUBSCRIPTIONERROR message.
* ----------
*/
static void
-pgstat_recv_subworker_error(PgStat_MsgSubWorkerError *msg, int len)
+pgstat_recv_subscription_error(PgStat_MsgSubscriptionError *msg, int len)
{
PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *subwentry;
+ PgStat_StatSubEntry *subentry;
dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
- /* Get the subscription worker stats */
- subwentry = pgstat_get_subworker_entry(dbentry, msg->m_subid,
- msg->m_subrelid, true);
- Assert(subwentry);
+ /* Get the subscription stats */
+ subentry = pgstat_get_sub_entry(dbentry, msg->m_subid, true);
+ Assert(subentry);
- if (subwentry->last_error_relid == msg->m_relid &&
- subwentry->last_error_command == msg->m_command &&
- subwentry->last_error_xid == msg->m_xid &&
- strcmp(subwentry->last_error_message, msg->m_message) == 0)
- {
- /*
- * The same error occurred again in succession, just update its
- * timestamp and count.
- */
- subwentry->last_error_count++;
- subwentry->last_error_time = msg->m_timestamp;
- return;
- }
-
- /* Otherwise, update the error information */
- subwentry->last_error_relid = msg->m_relid;
- subwentry->last_error_command = msg->m_command;
- subwentry->last_error_xid = msg->m_xid;
- subwentry->last_error_count = 1;
- subwentry->last_error_time = msg->m_timestamp;
- strlcpy(subwentry->last_error_message, msg->m_message,
- PGSTAT_SUBWORKERERROR_MSGLEN);
+ if (msg->m_is_apply_error)
+ subentry->apply_error_count++;
+ else
+ subentry->sync_error_count++;
}
/* ----------
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 5d9acc6173..e63118abaa 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -3377,7 +3377,6 @@ void
ApplyWorkerMain(Datum main_arg)
{
int worker_slot = DatumGetInt32(main_arg);
- MemoryContext cctx = CurrentMemoryContext;
MemoryContext oldctx;
char originname[NAMEDATALEN];
XLogRecPtr origin_startpos;
@@ -3485,20 +3484,15 @@ ApplyWorkerMain(Datum main_arg)
}
PG_CATCH();
{
- MemoryContext ecxt = MemoryContextSwitchTo(cctx);
- ErrorData *errdata = CopyErrorData();
-
/*
- * Report the table sync error. There is no corresponding message
- * type for table synchronization.
+ * Abort the current transaction so that we send the stats message
+ * in an idle state.
*/
- pgstat_report_subworker_error(MyLogicalRepWorker->subid,
- MyLogicalRepWorker->relid,
- MyLogicalRepWorker->relid,
- 0, /* message type */
- InvalidTransactionId,
- errdata->message);
- MemoryContextSwitchTo(ecxt);
+ AbortOutOfAnyTransaction();
+
+ /* Report the worker failed during table synchronization */
+ pgstat_report_subscription_error(MyLogicalRepWorker->subid, false);
+
PG_RE_THROW();
}
PG_END_TRY();
@@ -3625,22 +3619,14 @@ ApplyWorkerMain(Datum main_arg)
}
PG_CATCH();
{
- /* report the apply error */
- if (apply_error_callback_arg.command != 0)
- {
- MemoryContext ecxt = MemoryContextSwitchTo(cctx);
- ErrorData *errdata = CopyErrorData();
-
- pgstat_report_subworker_error(MyLogicalRepWorker->subid,
- MyLogicalRepWorker->relid,
- apply_error_callback_arg.rel != NULL
- ? apply_error_callback_arg.rel->localreloid
- : InvalidOid,
- apply_error_callback_arg.command,
- apply_error_callback_arg.remote_xid,
- errdata->message);
- MemoryContextSwitchTo(ecxt);
- }
+ /*
+ * Abort the current transaction so that we send the stats message in
+ * an idle state.
+ */
+ AbortOutOfAnyTransaction();
+
+ /* Report the worker failed during the application of the change */
+ pgstat_report_subscription_error(MyLogicalRepWorker->subid, true);
PG_RE_THROW();
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 30e8dfa7c1..a4e446dd6e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2163,7 +2163,7 @@ pg_stat_reset_single_table_counters(PG_FUNCTION_ARGS)
{
Oid taboid = PG_GETARG_OID(0);
- pgstat_reset_single_counter(taboid, InvalidOid, RESET_TABLE);
+ pgstat_reset_single_counter(taboid, RESET_TABLE);
PG_RETURN_VOID();
}
@@ -2173,38 +2173,21 @@ pg_stat_reset_single_function_counters(PG_FUNCTION_ARGS)
{
Oid funcoid = PG_GETARG_OID(0);
- pgstat_reset_single_counter(funcoid, InvalidOid, RESET_FUNCTION);
+ pgstat_reset_single_counter(funcoid, RESET_FUNCTION);
PG_RETURN_VOID();
}
Datum
-pg_stat_reset_subscription_worker_subrel(PG_FUNCTION_ARGS)
+pg_stat_reset_single_subscription_counters(PG_FUNCTION_ARGS)
{
Oid subid = PG_GETARG_OID(0);
- Oid relid = PG_ARGISNULL(1) ? InvalidOid : PG_GETARG_OID(1);
- pgstat_reset_single_counter(subid, relid, RESET_SUBWORKER);
+ pgstat_reset_single_counter(subid, RESET_SUBSCRIPTION);
PG_RETURN_VOID();
}
-/* Reset all subscription worker stats associated with the given subscription */
-Datum
-pg_stat_reset_subscription_worker_sub(PG_FUNCTION_ARGS)
-{
- Oid subid = PG_GETARG_OID(0);
-
- /*
- * Use subscription drop message to remove statistics of all subscription
- * workers.
- */
- pgstat_report_subscription_drop(subid);
-
- PG_RETURN_VOID();
-}
-
-
/* Reset SLRU counters (a specific one or all of them). */
Datum
pg_stat_reset_slru(PG_FUNCTION_ARGS)
@@ -2400,97 +2383,50 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
}
/*
- * Get the subscription worker statistics for the given subscription
- * (and relation).
+ * Get the subscription activity statistics for the given subscription.
*/
Datum
-pg_stat_get_subscription_worker(PG_FUNCTION_ARGS)
+pg_stat_get_subscription_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_SUBSCRIPTION_WORKER_COLS 8
+#define PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS 3
Oid subid = PG_GETARG_OID(0);
- Oid subrelid;
TupleDesc tupdesc;
- Datum values[PG_STAT_GET_SUBSCRIPTION_WORKER_COLS];
- bool nulls[PG_STAT_GET_SUBSCRIPTION_WORKER_COLS];
- PgStat_StatSubWorkerEntry *wentry;
- int i;
-
- if (PG_ARGISNULL(1))
- subrelid = InvalidOid;
- else
- subrelid = PG_GETARG_OID(1);
+ Datum values[PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS];
+ bool nulls[PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS];
+ PgStat_StatSubEntry *entry;
- /* Get subscription worker stats */
- wentry = pgstat_fetch_stat_subworker_entry(subid, subrelid);
-
- /* Return NULL if there is no worker statistics */
- if (wentry == NULL)
- PG_RETURN_NULL();
+ /* Get subscription stats */
+ entry = pgstat_fetch_stat_subentry(subid);
/* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_WORKER_COLS);
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "subid",
OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subrelid",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 3, "last_error_relid",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 4, "last_error_command",
- TEXTOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 5, "last_error_xid",
- XIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 6, "last_error_count",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "apply_error_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "sync_error_count",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 7, "last_error_message",
- TEXTOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 8, "last_error_time",
- TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
/* Initialise values and NULL flags arrays */
MemSet(values, 0, sizeof(values));
MemSet(nulls, 0, sizeof(nulls));
- i = 0;
/* subid */
- values[i++] = ObjectIdGetDatum(subid);
-
- /* subrelid */
- if (OidIsValid(subrelid))
- values[i++] = ObjectIdGetDatum(subrelid);
- else
- nulls[i++] = true;
-
- /* last_error_relid */
- if (OidIsValid(wentry->last_error_relid))
- values[i++] = ObjectIdGetDatum(wentry->last_error_relid);
- else
- nulls[i++] = true;
-
- /* last_error_command */
- if (wentry->last_error_command != 0)
- values[i++] =
- CStringGetTextDatum(logicalrep_message_type(wentry->last_error_command));
- else
- nulls[i++] = true;
-
- /* last_error_xid */
- if (TransactionIdIsValid(wentry->last_error_xid))
- values[i++] = TransactionIdGetDatum(wentry->last_error_xid);
- else
- nulls[i++] = true;
+ values[0] = ObjectIdGetDatum(subid);
- /* last_error_count */
- values[i++] = Int64GetDatum(wentry->last_error_count);
-
- /* last_error_message */
- values[i++] = CStringGetTextDatum(wentry->last_error_message);
+ /*
+ * Fill the subscription statistics. Return all-zero stats if there is no
+ * subscription statistics entry.
+ */
+ if (entry != NULL)
+ {
+ /* apply_error_count */
+ values[1] = Int64GetDatum(entry->apply_error_count);
- /* last_error_time */
- if (wentry->last_error_time != 0)
- values[i++] = TimestampTzGetDatum(wentry->last_error_time);
- else
- nulls[i++] = true;
+ /* sync_error_count */
+ values[2] = Int64GetDatum(entry->sync_error_count);
+ }
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7f1ee97f55..6f8fa91b37 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5377,14 +5377,14 @@
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
-{ oid => '8523', descr => 'statistics: information about subscription worker',
- proname => 'pg_stat_get_subscription_worker', prorows => '1', proisstrict => 'f',
- proretset => 't', provolatile => 's', proparallel => 'r',
- prorettype => 'record', proargtypes => 'oid oid',
- proallargtypes => '{oid,oid,oid,oid,oid,text,xid,int8,text,timestamptz}',
- proargmodes => '{i,i,o,o,o,o,o,o,o,o}',
- proargnames => '{subid,subrelid,subid,subrelid,last_error_relid,last_error_command,last_error_xid,last_error_count,last_error_message,last_error_time}',
- prosrc => 'pg_stat_get_subscription_worker' },
+{ oid => '8523', descr => 'statistics: information about subscription activity',
+ proname => 'pg_stat_get_subscription_activity', proisstrict => 'f',
+ provolatile => 's', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8}',
+ proargmodes => '{i,o,o,o}',
+ proargnames => '{subid,subid,apply_error_count,sync_error_count}',
+ prosrc => 'pg_stat_get_subscription_activity' },
{ oid => '6118', descr => 'statistics: information about subscription',
proname => 'pg_stat_get_subscription', prorows => '10', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
@@ -5773,15 +5773,10 @@
provolatile => 'v', prorettype => 'void', proargtypes => 'text',
prosrc => 'pg_stat_reset_replication_slot' },
{ oid => '8524',
- descr => 'statistics: reset collected statistics for a single subscription worker',
- proname => 'pg_stat_reset_subscription_worker', proisstrict => 'f',
- provolatile => 'v', prorettype => 'void', proargtypes => 'oid oid',
- prosrc => 'pg_stat_reset_subscription_worker_subrel' },
-{ oid => '8525',
- descr => 'statistics: reset all collected statistics for a single subscription',
- proname => 'pg_stat_reset_subscription_worker',
+ descr => 'statistics: reset collected statistics for a single subscription',
+ proname => 'pg_stat_reset_single_subscription_counters',
provolatile => 'v', prorettype => 'void', proargtypes => 'oid',
- prosrc => 'pg_stat_reset_subscription_worker_sub' },
+ prosrc => 'pg_stat_reset_single_subscription_counters' },
{ oid => '3163', descr => 'current trigger depth',
proname => 'pg_trigger_depth', provolatile => 's', proparallel => 'r',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index e10d20222a..9fae64e15e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -84,8 +84,8 @@ typedef enum StatMsgType
PGSTAT_MTYPE_REPLSLOT,
PGSTAT_MTYPE_CONNECT,
PGSTAT_MTYPE_DISCONNECT,
+ PGSTAT_MTYPE_SUBSCRIPTIONERROR,
PGSTAT_MTYPE_SUBSCRIPTIONPURGE,
- PGSTAT_MTYPE_SUBWORKERERROR,
} StatMsgType;
/* ----------
@@ -149,7 +149,7 @@ typedef enum PgStat_Single_Reset_Type
{
RESET_TABLE,
RESET_FUNCTION,
- RESET_SUBWORKER
+ RESET_SUBSCRIPTION
} PgStat_Single_Reset_Type;
/* ------------------------------------------------------------
@@ -368,7 +368,6 @@ typedef struct PgStat_MsgResetsinglecounter
Oid m_databaseid;
PgStat_Single_Reset_Type m_resettype;
Oid m_objectid;
- Oid m_subobjectid;
} PgStat_MsgResetsinglecounter;
/* ----------
@@ -558,37 +557,18 @@ typedef struct PgStat_MsgSubscriptionPurge
} PgStat_MsgSubscriptionPurge;
/* ----------
- * PgStat_MsgSubWorkerError Sent by the apply worker or the table sync
- * worker to report the error occurred while
- * processing changes.
+ * PgStat_MsgSubscriptionError Sent by the apply worker or the table sync
+ * worker to report an error on the subscription.
* ----------
*/
-#define PGSTAT_SUBWORKERERROR_MSGLEN 256
-typedef struct PgStat_MsgSubWorkerError
+typedef struct PgStat_MsgSubscriptionError
{
PgStat_MsgHdr m_hdr;
- /*
- * m_subid and m_subrelid are used to determine the subscription and the
- * reporter of the error. m_subrelid is InvalidOid if reported by an apply
- * worker otherwise reported by a table sync worker.
- */
Oid m_databaseid;
Oid m_subid;
- Oid m_subrelid;
-
- /*
- * Oid of the table that the reporter was actually processing. m_relid can
- * be InvalidOid if an error occurred during worker applying a
- * non-data-modification message such as RELATION.
- */
- Oid m_relid;
-
- LogicalRepMsgType m_command;
- TransactionId m_xid;
- TimestampTz m_timestamp;
- char m_message[PGSTAT_SUBWORKERERROR_MSGLEN];
-} PgStat_MsgSubWorkerError;
+ bool m_is_apply_error;
+} PgStat_MsgSubscriptionError;
/* ----------
* PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
@@ -767,8 +747,8 @@ typedef union PgStat_Msg
PgStat_MsgReplSlot msg_replslot;
PgStat_MsgConnect msg_connect;
PgStat_MsgDisconnect msg_disconnect;
+ PgStat_MsgSubscriptionError msg_subscriptionerror;
PgStat_MsgSubscriptionPurge msg_subscriptionpurge;
- PgStat_MsgSubWorkerError msg_subworkererror;
} PgStat_Msg;
@@ -823,16 +803,12 @@ typedef struct PgStat_StatDBEntry
TimestampTz stats_timestamp; /* time of db stats file update */
/*
- * tables, functions, and subscription workers must be last in the struct,
- * because we don't write the pointers out to the stats file.
- *
- * subworkers is the hash table of PgStat_StatSubWorkerEntry which stores
- * statistics of logical replication workers: apply worker and table sync
- * worker.
+ * tables, functions, and subscription must be last in the struct, because
+ * we don't write the pointers out to the stats file.
*/
HTAB *tables;
HTAB *functions;
- HTAB *subworkers;
+ HTAB *subscriptions;
} PgStat_StatDBEntry;
@@ -887,6 +863,16 @@ typedef struct PgStat_StatFuncEntry
PgStat_Counter f_self_time;
} PgStat_StatFuncEntry;
+/*
+ * Subscription statistics kept in the stats collector.
+ */
+typedef struct PgStat_StatSubEntry
+{
+ Oid subid; /* hash key (must be first) */
+
+ PgStat_Counter apply_error_count;
+ PgStat_Counter sync_error_count;
+} PgStat_StatSubEntry;
/*
* Archiver statistics kept in the stats collector
@@ -989,39 +975,6 @@ typedef struct PgStat_StatReplSlotEntry
TimestampTz stat_reset_timestamp;
} PgStat_StatReplSlotEntry;
-/* The lookup key for subscription worker hash table */
-typedef struct PgStat_StatSubWorkerKey
-{
- Oid subid;
-
- /*
- * Oid of the table for which tablesync worker will copy the initial data.
- * An InvalidOid will be assigned for apply workers.
- */
- Oid subrelid;
-} PgStat_StatSubWorkerKey;
-
-/*
- * Logical replication apply worker and table sync worker statistics kept in the
- * stats collector.
- */
-typedef struct PgStat_StatSubWorkerEntry
-{
- PgStat_StatSubWorkerKey key; /* hash key (must be first) */
-
- /*
- * Subscription worker error statistics representing an error that
- * occurred during application of changes or the initial table
- * synchronization.
- */
- Oid last_error_relid;
- LogicalRepMsgType last_error_command;
- TransactionId last_error_xid;
- PgStat_Counter last_error_count;
- TimestampTz last_error_time;
- char last_error_message[PGSTAT_SUBWORKERERROR_MSGLEN];
-} PgStat_StatSubWorkerEntry;
-
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
@@ -1111,8 +1064,7 @@ extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
extern void pgstat_reset_shared_counters(const char *);
-extern void pgstat_reset_single_counter(Oid objectid, Oid subobjectid,
- PgStat_Single_Reset_Type type);
+extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_reset_slru_counter(const char *);
extern void pgstat_reset_replslot_counter(const char *name);
@@ -1131,9 +1083,7 @@ extern void pgstat_report_checksum_failure(void);
extern void pgstat_report_replslot(const PgStat_StatReplSlotEntry *repSlotStat);
extern void pgstat_report_replslot_create(const char *slotname);
extern void pgstat_report_replslot_drop(const char *slotname);
-extern void pgstat_report_subworker_error(Oid subid, Oid subrelid, Oid relid,
- LogicalRepMsgType command,
- TransactionId xid, const char *errmsg);
+extern void pgstat_report_subscription_error(Oid subid, bool is_apply_error);
extern void pgstat_report_subscription_drop(Oid subid);
extern void pgstat_initialize(void);
@@ -1226,8 +1176,7 @@ extern void pgstat_send_wal(bool force);
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
-extern PgStat_StatSubWorkerEntry *pgstat_fetch_stat_subworker_entry(Oid subid,
- Oid subrelid);
+extern PgStat_StatSubEntry *pgstat_fetch_stat_subentry(Oid subid);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_BgWriterStats *pgstat_fetch_stat_bgwriter(void);
extern PgStat_CheckpointerStats *pgstat_fetch_stat_checkpointer(void);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 1420288d67..d24180f9b7 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2072,24 +2072,12 @@ pg_stat_subscription| SELECT su.oid AS subid,
st.latest_end_time
FROM (pg_subscription su
LEFT JOIN pg_stat_get_subscription(NULL::oid) st(subid, relid, pid, received_lsn, last_msg_send_time, last_msg_receipt_time, latest_end_lsn, latest_end_time) ON ((st.subid = su.oid)));
-pg_stat_subscription_workers| SELECT w.subid,
+pg_stat_subscription_activity| SELECT a.subid,
s.subname,
- w.subrelid,
- w.last_error_relid,
- w.last_error_command,
- w.last_error_xid,
- w.last_error_count,
- w.last_error_message,
- w.last_error_time
- FROM ( SELECT pg_subscription.oid AS subid,
- NULL::oid AS relid
- FROM pg_subscription
- UNION ALL
- SELECT pg_subscription_rel.srsubid AS subid,
- pg_subscription_rel.srrelid AS relid
- FROM pg_subscription_rel) sr,
- (LATERAL pg_stat_get_subscription_worker(sr.subid, sr.relid) w(subid, subrelid, last_error_relid, last_error_command, last_error_xid, last_error_count, last_error_message, last_error_time)
- JOIN pg_subscription s ON ((w.subid = s.oid)));
+ a.apply_error_count,
+ a.sync_error_count
+ FROM pg_subscription s,
+ LATERAL pg_stat_get_subscription_activity(s.oid) a(subid, apply_error_count, sync_error_count);
pg_stat_sys_indexes| SELECT pg_stat_all_indexes.relid,
pg_stat_all_indexes.indexrelid,
pg_stat_all_indexes.schemaname,
diff --git a/src/test/subscription/t/026_worker_stats.pl b/src/test/subscription/t/026_worker_stats.pl
index f72e4766e8..de706ca818 100644
--- a/src/test/subscription/t/026_worker_stats.pl
+++ b/src/test/subscription/t/026_worker_stats.pl
@@ -1,45 +1,13 @@
# Copyright (c) 2021-2022, PostgreSQL Global Development Group
-# Tests for subscription error stats.
+# Tests for subscription stats.
use strict;
use warnings;
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
use Test::More;
-# Test if the error reported on pg_stat_subscription_workers view is expected.
-sub test_subscription_error
-{
- my ($node, $relname, $command, $xid, $by_apply_worker, $errmsg_prefix, $msg)
- = @_;
-
- my $check_sql = qq[
-SELECT count(1) > 0
-FROM pg_stat_subscription_workers
-WHERE last_error_relid = '$relname'::regclass
- AND starts_with(last_error_message, '$errmsg_prefix')];
-
- # subrelid
- $check_sql .= $by_apply_worker
- ? qq[ AND subrelid IS NULL]
- : qq[ AND subrelid = '$relname'::regclass];
-
- # last_error_command
- $check_sql .= $command eq ''
- ? qq[ AND last_error_command IS NULL]
- : qq[ AND last_error_command = '$command'];
-
- # last_error_xid
- $check_sql .= $xid eq ''
- ? qq[ AND last_error_xid IS NULL]
- : qq[ AND last_error_xid = '$xid'::xid];
-
- # Wait for the particular error statistics to be reported.
- $node->poll_query_until('postgres', $check_sql,
-) or die "Timed out while waiting for " . $msg;
-}
-
# Create publisher node.
my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
@@ -48,116 +16,83 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
-
-# The subscriber will enter an infinite error loop, so we don't want
-# to overflow the server log with error messages.
-$node_subscriber->append_conf('postgresql.conf',
- qq[
-wal_retrieve_retry_interval = 2s
-]);
$node_subscriber->start;
# Initial table setup on both publisher and subscriber. On subscriber we
# create the same tables but with primary keys. Also, insert some data that
# will conflict with the data replicated from publisher later.
$node_publisher->safe_psql(
- 'postgres',
- qq[
+ 'postgres',
+ qq[
BEGIN;
CREATE TABLE test_tab1 (a int);
-CREATE TABLE test_tab2 (a int);
INSERT INTO test_tab1 VALUES (1);
-INSERT INTO test_tab2 VALUES (1);
COMMIT;
]);
$node_subscriber->safe_psql(
- 'postgres',
- qq[
+ 'postgres',
+ qq[
BEGIN;
CREATE TABLE test_tab1 (a int primary key);
-CREATE TABLE test_tab2 (a int primary key);
-INSERT INTO test_tab2 VALUES (1);
+INSERT INTO test_tab1 VALUES (1);
COMMIT;
]);
-# Setup publications.
+# Setup publication.
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
-$node_publisher->safe_psql(
- 'postgres',
- "CREATE PUBLICATION tap_pub FOR TABLE test_tab1, test_tab2;");
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION tap_pub FOR TABLE test_tab1;");
# There shouldn't be any subscription errors before starting logical replication.
-my $result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(1) FROM pg_stat_subscription_workers");
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(1) FROM pg_stat_subscription_activity");
is($result, qq(0), 'check no subscription error');
-# Create subscription. The table sync for test_tab2 on tap_sub will enter into
+# Create subscription. The table sync for test_tab1 on tap_sub will enter into
# infinite error loop due to violating the unique constraint.
-$node_subscriber->safe_psql(
- 'postgres',
- "CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub;");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub;"
+);
$node_publisher->wait_for_catchup('tap_sub');
+# Wait for the table sync error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT apply_error_count = 0 AND sync_error_count > 0
+FROM pg_stat_subscription_activity
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for table sync error";
+
+# Truncate test_tab1 so that table sync can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
+
# Wait for initial table sync for test_tab1 to finish.
$node_subscriber->poll_query_until(
- 'postgres',
- qq[
+ 'postgres',
+ qq[
SELECT count(1) = 1 FROM pg_subscription_rel
WHERE srrelid = 'test_tab1'::regclass AND srsubstate in ('r', 's')
]) or die "Timed out while waiting for subscriber to synchronize data";
-# Check the initial data.
-$result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(a) FROM test_tab1");
-is($result, q(1), 'check initial data are copied to subscriber');
-
-# Insert more data to test_tab1, raising an error on the subscriber due to
-# violation of the unique constraint on test_tab1.
-my $xid = $node_publisher->safe_psql(
- 'postgres',
- qq[
-BEGIN;
-INSERT INTO test_tab1 VALUES (1);
-SELECT pg_current_xact_id()::xid;
-COMMIT;
-]);
-test_subscription_error($node_subscriber, 'test_tab1', 'INSERT', $xid,
- 1, # check apply worker error
- qq(duplicate key value violates unique constraint),
- 'error reported by the apply worker');
-
-# Check the table sync worker's error in the view.
-test_subscription_error($node_subscriber, 'test_tab2', '', '',
- 0, # check tablesync worker error
- qq(duplicate key value violates unique constraint),
- 'the error reported by the table sync worker');
-
-# Test for resetting subscription worker statistics.
-# Truncate test_tab1 and test_tab2 so that applying changes and table sync can
-# continue, respectively.
-$node_subscriber->safe_psql(
- 'postgres',
- "TRUNCATE test_tab1, test_tab2;");
+# Insert more data to test_tab1 on the subscriber and then on the publisher, raising an
+# error on the subscriber due to violation of the unique constraint on test_tab1.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_tab1 VALUES (2)");
-# Wait for the data to be replicated.
-$node_subscriber->poll_query_until(
- 'postgres',
- "SELECT count(1) > 0 FROM test_tab1");
-$node_subscriber->poll_query_until(
- 'postgres',
- "SELECT count(1) > 0 FROM test_tab2");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_tab1 VALUES (1)");
-# There shouldn't be any errors in the view after dropping the subscription.
-$node_subscriber->safe_psql(
- 'postgres',
- "DROP SUBSCRIPTION tap_sub;");
-$result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(1) FROM pg_stat_subscription_workers");
-is($result, q(0), 'no error after dropping subscription');
+# Wait for the apply error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT apply_error_count > 0 AND sync_error_count > 0
+FROM pg_stat_subscription_activity
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for apply error";
+
+# Truncate test_tab1 so that table sync can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
$node_subscriber->stop('fast');
$node_publisher->stop('fast');
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 15684f53ba..94b099df5e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1944,8 +1944,8 @@ PgStat_MsgResetsharedcounter
PgStat_MsgResetsinglecounter
PgStat_MsgResetslrucounter
PgStat_MsgSLRU
+PgStat_MsgSubscriptionError
PgStat_MsgSubscriptionPurge
-PgStat_MsgSubWorkerError
PgStat_MsgTabpurge
PgStat_MsgTabstat
PgStat_MsgTempFile
@@ -1957,8 +1957,7 @@ PgStat_Single_Reset_Type
PgStat_StatDBEntry
PgStat_StatFuncEntry
PgStat_StatReplSlotEntry
-PgStat_StatSubWorkerEntry
-PgStat_StatSubWorkerKey
+PgStat_StatSubEntry
PgStat_StatTabEntry
PgStat_SubXactStatus
PgStat_TableCounts
--
2.24.3 (Apple Git-128)
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-22 09:53 Amit Kapila <[email protected]>
parent: Masahiko Sawada <[email protected]>
4 siblings, 1 reply; 42+ messages in thread
From: Amit Kapila @ 2022-02-22 09:53 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Tue, Feb 22, 2022 at 11:15 AM Masahiko Sawada <[email protected]> wrote:
>
> I've attached a patch that changes pg_stat_subscription_workers view.
> It removes non-cumulative values such as error details such as
> error-XID and the error message from the view, and consequently the
> view now has only cumulative statistics counters: apply_error_count
> and sync_error_count. Since the new view name is under discussion I
> temporarily chose pg_stat_subscription_activity.
>
Few comments:
=============
1.
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -637,11 +637,9 @@ REVOKE EXECUTE ON FUNCTION
pg_stat_reset_single_table_counters(oid) FROM public;
REVOKE EXECUTE ON FUNCTION
pg_stat_reset_single_function_counters(oid) FROM public;
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_replication_slot(text) FROM public;
-
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid) FROM public;
+REVOKE EXECUTE ON FUNCTION
pg_stat_reset_single_subscription_counters(oid) FROM public;
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid,
oid) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_stat_reset_replication_slot(text) FROM public;
Is there a need to change anything about
pg_stat_reset_replication_slot() in this patch?
2. Do we still need to use LATERAL in the view's query?
3. Can we send error stats pgstat_report_stat() as that will be called
via proc_exit() path. We can set the phase (apply/sync) in
apply_error_callback_arg and then use that to send the appropriate
message. I think this will obviate the need for try..catch.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 42+ messages in thread
* RE: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-22 12:22 [email protected] <[email protected]>
parent: Masahiko Sawada <[email protected]>
4 siblings, 1 reply; 42+ messages in thread
From: [email protected] @ 2022-02-22 12:22 UTC (permalink / raw)
To: 'Masahiko Sawada' <[email protected]>; Amit Kapila <[email protected]>; +Cc: Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Tuesday, February 22, 2022 2:45 PM Masahiko Sawada <[email protected]> wrote:
> I've attached a patch that changes pg_stat_subscription_workers view.
> It removes non-cumulative values such as error details such as error-XID and
> the error message from the view, and consequently the view now has only
> cumulative statistics counters: apply_error_count and sync_error_count. Since
> the new view name is under discussion I temporarily chose
> pg_stat_subscription_activity.
Hi, thank you for sharing the patch.
Few minor comments for v1.
(1) commit message's typo
This commits changes the view so that it stores only statistics
counters: apply_error_count and sync_error_count.
"This commits" -> "This commit"
(2) minor improvement suggestion for the commit message
I suggest that we touch the commit id 8d74fc9
that introduces the pg_stat_subscription_workers
in the commit message, for better traceability. Below is an example.
From:
As the result of the discussion, we've concluded that the stats
collector is not an appropriate place to store the error information of
subscription workers.
To:
As the result of the discussion about the view introduced by 8d74fc9,...
(3) doc/src/sgml/logical-replication.sgml
Kindly refer to commit id 85c61ba for the detail.
You forgot "the" in the below sentence.
@@ -346,8 +346,6 @@
<para>
A conflict will produce an error and will stop the replication; it must be
resolved manually by the user. Details about the conflict can be found in
- <link linkend="monitoring-pg-stat-subscription-workers">
- <structname>pg_stat_subscription_workers</structname></link> and the
subscriber's server log.
</para>
From:
subscriber's server log.
to:
the subscriber's server log.
(4) doc/src/sgml/monitoring.sgml
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_time</structfield> <type>timestamp with time zone</type>
+ <structfield>sync_error_count</structfield> <type>uint8</type>
</para>
<para>
- Last time at which this error occurred
+ Number of times the error occurred during the initial data copy
</para></entry>
I supposed it might be better to use "initial data sync"
or "initial data synchronization", rather than "initial data copy".
(5) src/test/subscription/t/026_worker_stats.pl
+# Truncate test_tab1 so that table sync can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
The second truncate is for apply, isn't it? Therefore, kindly change
From:
Truncate test_tab1 so that table sync can continue.
To:
Truncate test_tab1 so that apply can continue.
(6) src/test/subscription/t/026_worker_stats.pl
+# Insert more data to test_tab1 on the subscriber and then on the publisher, raising an
+# error on the subscriber due to violation of the unique constraint on test_tab1.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_tab1 VALUES (2)");
Did we need this insert ?
If you want to indicate the apply is working okay after the error of table sync is solved,
waiting for the max value in the test_tab1 becoming 2 on the subscriber by polling query
would work. But, I was not sure if this is essentially necessary for the testing purpose.
Best Regards,
Takamichi Osumi
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-22 14:32 Masahiko Sawada <[email protected]>
parent: Amit Kapila <[email protected]>
0 siblings, 1 reply; 42+ messages in thread
From: Masahiko Sawada @ 2022-02-22 14:32 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Tue, Feb 22, 2022 at 6:53 PM Amit Kapila <[email protected]> wrote:
>
> On Tue, Feb 22, 2022 at 11:15 AM Masahiko Sawada <[email protected]> wrote:
> >
> > I've attached a patch that changes pg_stat_subscription_workers view.
> > It removes non-cumulative values such as error details such as
> > error-XID and the error message from the view, and consequently the
> > view now has only cumulative statistics counters: apply_error_count
> > and sync_error_count. Since the new view name is under discussion I
> > temporarily chose pg_stat_subscription_activity.
> >
>
> Few comments:
> =============
Thank you for the comments!
> 1.
> --- a/src/backend/catalog/system_functions.sql
> +++ b/src/backend/catalog/system_functions.sql
> @@ -637,11 +637,9 @@ REVOKE EXECUTE ON FUNCTION
> pg_stat_reset_single_table_counters(oid) FROM public;
>
> REVOKE EXECUTE ON FUNCTION
> pg_stat_reset_single_function_counters(oid) FROM public;
>
> -REVOKE EXECUTE ON FUNCTION pg_stat_reset_replication_slot(text) FROM public;
> -
> -REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid) FROM public;
> +REVOKE EXECUTE ON FUNCTION
> pg_stat_reset_single_subscription_counters(oid) FROM public;
>
> -REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid,
> oid) FROM public;
> +REVOKE EXECUTE ON FUNCTION pg_stat_reset_replication_slot(text) FROM public;
>
> Is there a need to change anything about
> pg_stat_reset_replication_slot() in this patch?
It doesn't change pg_stat_reset_replication_slot() but just changes
the order in order to put the modified function
pg_stat_reset_single_subscription_counters() closer to other similar
functions such as pg_stat_reset_single_function_counters().
>
> 2. Do we still need to use LATERAL in the view's query?
There are some functions that use LATERAL in a similar way but it
seems no need to put LATERAL before the function call. Will remove.
> 3. Can we send error stats pgstat_report_stat() as that will be called
> via proc_exit() path. We can set the phase (apply/sync) in
> apply_error_callback_arg and then use that to send the appropriate
> message. I think this will obviate the need for try..catch.
If we use pgstat_report_stat() to send subscription stats messages,
all processes end up going through that path. It might not bring
overhead in practice but I'd like to avoid it. And, since the apply
worker also calls pgstat_report_stat() at the end of the transaction,
we might need to change pgstat_report_stat() so that it doesn't send
the subscription messages when it gets called at the end of the
transaction. I think it's likely that PG_TRY() and PG_CATCH() wil be
added for example, when the disable_on_error feature or the storing
error details feature is introduced, so obviating the need for them at
this point would not benefit much.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-22 14:46 Masahiko Sawada <[email protected]>
parent: [email protected] <[email protected]>
0 siblings, 1 reply; 42+ messages in thread
From: Masahiko Sawada @ 2022-02-22 14:46 UTC (permalink / raw)
To: [email protected] <[email protected]>; +Cc: Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Tue, Feb 22, 2022 at 9:22 PM [email protected]
<[email protected]> wrote:
>
> On Tuesday, February 22, 2022 2:45 PM Masahiko Sawada <[email protected]> wrote:
> > I've attached a patch that changes pg_stat_subscription_workers view.
> > It removes non-cumulative values such as error details such as error-XID and
> > the error message from the view, and consequently the view now has only
> > cumulative statistics counters: apply_error_count and sync_error_count. Since
> > the new view name is under discussion I temporarily chose
> > pg_stat_subscription_activity.
> Hi, thank you for sharing the patch.
>
>
> Few minor comments for v1.
Thank you for the comments!
>
> (1) commit message's typo
>
> This commits changes the view so that it stores only statistics
> counters: apply_error_count and sync_error_count.
>
> "This commits" -> "This commit"
Will fix.
>
> (2) minor improvement suggestion for the commit message
>
> I suggest that we touch the commit id 8d74fc9
> that introduces the pg_stat_subscription_workers
> in the commit message, for better traceability. Below is an example.
>
> From:
> As the result of the discussion, we've concluded that the stats
> collector is not an appropriate place to store the error information of
> subscription workers.
>
> To:
> As the result of the discussion about the view introduced by 8d74fc9,...
Okay, will add the commit reference.
>
> (3) doc/src/sgml/logical-replication.sgml
>
> Kindly refer to commit id 85c61ba for the detail.
> You forgot "the" in the below sentence.
>
> @@ -346,8 +346,6 @@
> <para>
> A conflict will produce an error and will stop the replication; it must be
> resolved manually by the user. Details about the conflict can be found in
> - <link linkend="monitoring-pg-stat-subscription-workers">
> - <structname>pg_stat_subscription_workers</structname></link> and the
> subscriber's server log.
> </para>
>
> From:
> subscriber's server log.
> to:
> the subscriber's server log.
Will fix.
>
> (4) doc/src/sgml/monitoring.sgml
>
> <row>
> <entry role="catalog_table_entry"><para role="column_definition">
> - <structfield>last_error_time</structfield> <type>timestamp with time zone</type>
> + <structfield>sync_error_count</structfield> <type>uint8</type>
> </para>
> <para>
> - Last time at which this error occurred
> + Number of times the error occurred during the initial data copy
> </para></entry>
>
> I supposed it might be better to use "initial data sync"
> or "initial data synchronization", rather than "initial data copy".
"Initial data synchronization" sounds like the whole table
synchronization process including COPY and applying changes to catch
up. But sync_error_count is incremented only during COPY so I used
"initial data copy". What do you think?
>
> (5) src/test/subscription/t/026_worker_stats.pl
>
> +# Truncate test_tab1 so that table sync can continue.
> +$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
>
> The second truncate is for apply, isn't it? Therefore, kindly change
>
> From:
> Truncate test_tab1 so that table sync can continue.
> To:
> Truncate test_tab1 so that apply can continue.
Right, will fix.
>
> (6) src/test/subscription/t/026_worker_stats.pl
>
> +# Insert more data to test_tab1 on the subscriber and then on the publisher, raising an
> +# error on the subscriber due to violation of the unique constraint on test_tab1.
> +$node_subscriber->safe_psql('postgres', "INSERT INTO test_tab1 VALUES (2)");
>
> Did we need this insert ?
> If you want to indicate the apply is working okay after the error of table sync is solved,
> waiting for the max value in the test_tab1 becoming 2 on the subscriber by polling query
> would work. But, I was not sure if this is essentially necessary for the testing purpose.
You're right, it's not necessary. Also, it seems better to change the
TAP test file name from 026_worker_stats.pl to 026_stats.pl. Will
incorporate these changes.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 42+ messages in thread
* RE: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-23 01:13 [email protected] <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: [email protected] @ 2022-02-23 01:13 UTC (permalink / raw)
To: 'Masahiko Sawada' <[email protected]>; +Cc: Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Tuesday, February 22, 2022 11:47 PM Masahiko Sawada <[email protected]> wrote:
> On Tue, Feb 22, 2022 at 9:22 PM [email protected]
> <[email protected]> wrote:
> > (4) doc/src/sgml/monitoring.sgml
> >
> > <row>
> > <entry role="catalog_table_entry"><para
> role="column_definition">
> > - <structfield>last_error_time</structfield> <type>timestamp with
> time zone</type>
> > + <structfield>sync_error_count</structfield> <type>uint8</type>
> > </para>
> > <para>
> > - Last time at which this error occurred
> > + Number of times the error occurred during the initial data
> > + copy
> > </para></entry>
> >
> > I supposed it might be better to use "initial data sync"
> > or "initial data synchronization", rather than "initial data copy".
>
> "Initial data synchronization" sounds like the whole table synchronization
> process including COPY and applying changes to catch up. But
> sync_error_count is incremented only during COPY so I used "initial data copy".
> What do you think?
Okay. Please keep it as is.
Best Regards,
Takamichi Osumi
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-23 02:14 Andres Freund <[email protected]>
parent: Masahiko Sawada <[email protected]>
4 siblings, 2 replies; 42+ messages in thread
From: Andres Freund @ 2022-02-23 02:14 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Amit Kapila <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Hi,
On 2022-02-22 14:45:19 +0900, Masahiko Sawada wrote:
> I've attached a patch that changes pg_stat_subscription_workers view.
Thanks for working on this!
Why are the stats stored in the per-database stats file / as a second level
below the database? While they're also associated with a database, it's a
global catalog, so it seems to make more sense to have them "live" globally as
well?
Not just from an aesthetical perspective, but there might also be cases where
it's useful to send stats from the stats launcher. E.g. the number of times
the launcher couldn't start a worker because the max numbers of workers was
already active or such.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-23 02:46 Masahiko Sawada <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 0 replies; 42+ messages in thread
From: Masahiko Sawada @ 2022-02-23 02:46 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Amit Kapila <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Wed, Feb 23, 2022 at 11:14 AM Andres Freund <[email protected]> wrote:
>
> Hi,
>
> On 2022-02-22 14:45:19 +0900, Masahiko Sawada wrote:
> > I've attached a patch that changes pg_stat_subscription_workers view.
>
> Thanks for working on this!
>
> Why are the stats stored in the per-database stats file / as a second level
> below the database? While they're also associated with a database, it's a
> global catalog, so it seems to make more sense to have them "live" globally as
> well?
Good point. The reason why we used to use per-database stats file is
that we were storing some relation information there. But now that we
don't need to have such information, it makes more sense to have them
live globally. I'll change the patch accordingly.
>
> Not just from an aesthetical perspective, but there might also be cases where
> it's useful to send stats from the stats launcher. E.g. the number of times
> the launcher couldn't start a worker because the max numbers of workers was
> already active or such.
Good idea.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 42+ messages in thread
* RE: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-23 03:00 [email protected] <[email protected]>
parent: Masahiko Sawada <[email protected]>
4 siblings, 0 replies; 42+ messages in thread
From: [email protected] @ 2022-02-23 03:00 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; Amit Kapila <[email protected]>; +Cc: Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Tue, Feb 22, 2022 1:45 PM Masahiko Sawada <[email protected]> wrote:
>
> I've attached a patch that changes pg_stat_subscription_workers view.
> It removes non-cumulative values such as error details such as
> error-XID and the error message from the view, and consequently the
> view now has only cumulative statistics counters: apply_error_count
> and sync_error_count. Since the new view name is under discussion I
> temporarily chose pg_stat_subscription_activity.
>
Thanks for your patch.
Few comments:
1.
+ <structfield>apply_error_count</structfield> <type>uint8</type>
...
+ <structfield>sync_error_count</structfield> <type>uint8</type>
It seems that Postgres has no data type named uint8, should we change it to
bigint?
2.
+# Wait for the table sync error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT apply_error_count = 0 AND sync_error_count > 0
+FROM pg_stat_subscription_activity
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for table sync error";
We want to check table sync error here, but do we need to check
"apply_error_count = 0"? I am not sure if it is possible that the apply worker has
an unexpected error, which would cause this test to fail.
Regards,
Tang
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-23 03:07 Peter Smith <[email protected]>
parent: Masahiko Sawada <[email protected]>
4 siblings, 2 replies; 42+ messages in thread
From: Peter Smith @ 2022-02-23 03:07 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Hi. Below are my review comments for the v1 patch.
======
1. Commit message - wording
As the result of the discussion, we've concluded that the stats
collector is not an appropriate place to store the error information of
subscription workers.
SUGGESTION
It was decided (refer to the Discussion link below) that the stats
collector is not an appropriate place to store the error information of
subscription workers.
~~~
2. Commit message - wording
This commits changes the view so that it stores only statistics
counters: apply_error_count and sync_error_count.
SUGGESTION
"This commits changes the view" --> "This patch changes the
pg_stat_subscription_workers view"
~~~
3. Commit message - wording
Removing these error details, since we don't need to record the
error information for apply workers and tablesync workers separately,
the view now has one entry per subscription.
DID THIS MEAN...
After removing these error details, there is no longer separate
information for apply workers and tablesync workers, so the view now
has only one entry per subscription.
--
But anyway, that is not entirely true either because those counters
are separate information for the different workers, right?
~~
4. Commit message - wording
Also, it changes the view name to pg_stat_subscription_activity
since the word "worker" is an implementation detail that we use one
worker for one tablesync.
SUGGESTION
"Also, it changes" --> "The patch also changes" ...
~~~
5. doc/src/sgml/monitoring.sgml - wording
<para>
- The <structname>pg_stat_subscription_workers</structname> view will contain
- one row per subscription worker on which errors have occurred, for workers
- applying logical replication changes and workers handling the initial data
- copy of the subscribed tables. The statistics entry is removed when the
+ The <structname>pg_stat_subscription_activity</structname> view will contain
+ one row per subscription. The statistics entry is removed when the
corresponding subscription is dropped.
</para>
SUGGESTION
"The statistics entry is removed" --> "This row is removed"
~~~
6. doc/src/sgml/monitoring.sgml - why two counters?
Please forgive this noob question...
I see there are 2 error_count columns (one for each kind of worker)
but I was wondering why it is useful for users to be able to
distinguish if the error came from the tablesync workers or from the
apply workers? Do you have any example?
Also, IIRC sometimes the tablesync might actually do a few "apply"
changes itself... so the distinction may become a bit fuzzy...
~~~
7. src/backend/postmaster/pgstat.c - comment
@@ -1313,13 +1312,13 @@ pgstat_vacuum_stat(void)
}
/*
- * Repeat for subscription workers. Similarly, we needn't bother in the
- * common case where no subscription workers' stats are being collected.
+ * Repeat for subscription. Similarly, we needn't bother in the common
+ * case where no subscription stats are being collected.
*/
typo?
"Repeat for subscription." --> "Repeat for subscriptions."
~~~
8. src/backend/postmaster/pgstat.c
@@ -3000,32 +2968,29 @@ pgstat_fetch_stat_funcentry(Oid func_id)
/*
* ---------
- * pgstat_fetch_stat_subworker_entry() -
+ * pgstat_fetch_stat_subentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for subscription worker or NULL.
+ * the collected statistics for subscription or NULL.
* ---------
*/
-PgStat_StatSubWorkerEntry *
-pgstat_fetch_stat_subworker_entry(Oid subid, Oid subrelid)
+PgStat_StatSubEntry *
+pgstat_fetch_stat_subentry(Oid subid)
There seems some kind of inconsistency because the member name is
called "subscriptions" but sometimes it seems singular.
Some places (e.g. pgstat_vacuum_stat) will iterate multiple results,
but then other places (like this function) just return to a single
"subscription" (or "entry").
I suspect all the code may be fine; probably it is just some
inconsistent (singular/plural) comments that have confused things a
bit.
~~~
9. src/backend/replication/logical/worker.c - subscription id
+ /* Report the worker failed during table synchronization */
+ pgstat_report_subscription_error(MyLogicalRepWorker->subid, false);
and
+ /* Report the worker failed during the application of the change */
+ pgstat_report_subscription_error(MyLogicalRepWorker->subid, true);
Why don't these use MySubscription->oid instead of MyLogicalRepWorker->subid?
~~~
10. src/include/pgstat.h - enum order
@@ -84,8 +84,8 @@ typedef enum StatMsgType
PGSTAT_MTYPE_REPLSLOT,
PGSTAT_MTYPE_CONNECT,
PGSTAT_MTYPE_DISCONNECT,
+ PGSTAT_MTYPE_SUBSCRIPTIONERROR,
PGSTAT_MTYPE_SUBSCRIPTIONPURGE,
- PGSTAT_MTYPE_SUBWORKERERROR,
} StatMsgType;
This change rearranges the enum order. Maybe it is safer not to do this?
~~~
11. src/include/pgstat.h
@@ -767,8 +747,8 @@ typedef union PgStat_Msg
PgStat_MsgReplSlot msg_replslot;
PgStat_MsgConnect msg_connect;
PgStat_MsgDisconnect msg_disconnect;
+ PgStat_MsgSubscriptionError msg_subscriptionerror;
PgStat_MsgSubscriptionPurge msg_subscriptionpurge;
- PgStat_MsgSubWorkerError msg_subworkererror;
} PgStat_Msg;
This change also rearranges the order. Maybe there was no good reason
to do that?
~~~
12. src/include/pgstat.h - PgStat_StatDBEntry
@@ -823,16 +803,12 @@ typedef struct PgStat_StatDBEntry
TimestampTz stats_timestamp; /* time of db stats file update */
/*
- * tables, functions, and subscription workers must be last in the struct,
- * because we don't write the pointers out to the stats file.
- *
- * subworkers is the hash table of PgStat_StatSubWorkerEntry which stores
- * statistics of logical replication workers: apply worker and table sync
- * worker.
+ * tables, functions, and subscription must be last in the struct, because
+ * we don't write the pointers out to the stats file.
*/
Should that say "tables, functions, and subscriptions" (plural)
------
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-23 03:47 Amit Kapila <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Amit Kapila @ 2022-02-23 03:47 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Tue, Feb 22, 2022 at 8:02 PM Masahiko Sawada <[email protected]> wrote:
>
> On Tue, Feb 22, 2022 at 6:53 PM Amit Kapila <[email protected]> wrote:
> >
>
> > 3. Can we send error stats pgstat_report_stat() as that will be called
> > via proc_exit() path. We can set the phase (apply/sync) in
> > apply_error_callback_arg and then use that to send the appropriate
> > message. I think this will obviate the need for try..catch.
>
> If we use pgstat_report_stat() to send subscription stats messages,
> all processes end up going through that path. It might not bring
> overhead in practice but I'd like to avoid it.
>
I am not sure about overhead but I see another problem if we use that
approach. In the exit path, logicalrep_worker_onexit() will get called
before pgstat_report_stat() and that will clear the
MyLogicalRepWorker->subid, so we won't know the id for which to send
stats. So, the way patch is doing seems reasonable to me unless
someone has better ideas.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-23 06:20 Peter Smith <[email protected]>
parent: Peter Smith <[email protected]>
1 sibling, 0 replies; 42+ messages in thread
From: Peter Smith @ 2022-02-23 06:20 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Below are my review comments just for the v1 patch test code.
======
1. "table sync error" versus "tablesync error"
+# Wait for the table sync error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT apply_error_count = 0 AND sync_error_count > 0
+FROM pg_stat_subscription_activity
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for table sync error";
+
+# Truncate test_tab1 so that table sync can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
+
# Wait for initial table sync for test_tab1 to finish.
IMO all these "table sync" should be changed to "tablesync", because a
table "sync error" sounds like something completely different to a
"tablesync error".
SUGGESTIONS
- "Wait for the table sync error to be reported." --> "Wait for the
tablesync error to be reported."
- "Timed out while waiting for table sync error" --> "Timed out while
waiting for tablesync error"
- "Truncate test_tab1 so that table sync can continue." --> "Truncate
test_tab1 so that tablesync worker can fun to completion."
- "Wait for initial table sync for test_tab1 to finish." --> "Wait for
the tablesync worker of test_tab1 to finish."
~~~
2. Unnecessary INSERT VALUES (2)?
(I think this is a duplicate of what [Osumi] #6 reported)
+# Insert more data to test_tab1 on the subscriber and then on the
publisher, raising an
+# error on the subscriber due to violation of the unique constraint
on test_tab1.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_tab1 VALUES (2)");
Why does the test do INSERT data (2)? There is already data (1) from
the tablesync which will cause an apply worker PK violation when
another VALUES (1) is published.
Note, the test comment also needs to change...
~~~
3. Wait for the apply worker error
+# Wait for the apply error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT apply_error_count > 0 AND sync_error_count > 0
+FROM pg_stat_subscription_activity
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for apply error";
This test is only for apply worker errors. So why is the test SQL
checking "AND sync_error_count > 0"?
(This is similar to what [Tang] #2 reported, but I think she was
referring to the other tablesync test)
~~~
4. Wrong worker?
(looks like a duplicate of what [Osumi] #5 already)
+
+# Truncate test_tab1 so that table sync can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
$node_subscriber->stop('fast');
$node_publisher->stop('fast');
Cut/paste error? Aren't you doing TRUNCATE here so the apply worker
can continue; not the tablesync worker (which already completed)
"Truncate test_tab1 so that table sync can continue." --> "Truncate
test_tab1 so that the apply worker can continue."
------
[Osumi] https://www.postgresql.org/message-id/CAD21AoBRt%3DcyKsZP83rcMkHnT498gHH0TEP34fZBrGCxT-Ahwg%40mail.g...
[Tang] https://www.postgresql.org/message-id/TYCPR01MB612840D018FEBD38268CC83BFB3C9%40TYCPR01MB6128.jpnprd0...
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 01:32 Masahiko Sawada <[email protected]>
parent: Peter Smith <[email protected]>
1 sibling, 2 replies; 42+ messages in thread
From: Masahiko Sawada @ 2022-02-24 01:32 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Hi,
On Wed, Feb 23, 2022 at 12:08 PM Peter Smith <[email protected]> wrote:
>
> Hi. Below are my review comments for the v1 patch.
Thank you for the comments! I've attached the latest version patch
that incorporated all comments I got so far. The primary change from
the previous version is that the subscription statistics live globally
rather than per-database.
>
> ======
>
> 1. Commit message - wording
>
> As the result of the discussion, we've concluded that the stats
> collector is not an appropriate place to store the error information of
> subscription workers.
>
> SUGGESTION
> It was decided (refer to the Discussion link below) that the stats
> collector is not an appropriate place to store the error information of
> subscription workers.
Fixed.
>
> ~~~
>
> 2. Commit message - wording
>
> This commits changes the view so that it stores only statistics
> counters: apply_error_count and sync_error_count.
>
> SUGGESTION
> "This commits changes the view" --> "This patch changes the
> pg_stat_subscription_workers view"
Fixed.
>
> ~~~
>
> 3. Commit message - wording
>
> Removing these error details, since we don't need to record the
> error information for apply workers and tablesync workers separately,
> the view now has one entry per subscription.
>
> DID THIS MEAN...
> After removing these error details, there is no longer separate
> information for apply workers and tablesync workers, so the view now
> has only one entry per subscription.
>
> --
>
> But anyway, that is not entirely true either because those counters
> are separate information for the different workers, right?
Right. Since I also made the subscription statistics a cluster-wide
statistics, I've changed this part accordingly.
>
> ~~
>
> 4. Commit message - wording
>
> Also, it changes the view name to pg_stat_subscription_activity
> since the word "worker" is an implementation detail that we use one
> worker for one tablesync.
>
> SUGGESTION
> "Also, it changes" --> "The patch also changes" ...
Fixed.
>
> ~~~
>
> 5. doc/src/sgml/monitoring.sgml - wording
>
> <para>
> - The <structname>pg_stat_subscription_workers</structname> view will contain
> - one row per subscription worker on which errors have occurred, for workers
> - applying logical replication changes and workers handling the initial data
> - copy of the subscribed tables. The statistics entry is removed when the
> + The <structname>pg_stat_subscription_activity</structname> view will contain
> + one row per subscription. The statistics entry is removed when the
> corresponding subscription is dropped.
> </para>
>
> SUGGESTION
> "The statistics entry is removed" --> "This row is removed"
On second thoughts, this sentence is not necessary since it's obvious
and descriptions of other stats view don't mention it.
>
> ~~~
>
> 6. doc/src/sgml/monitoring.sgml - why two counters?
>
> Please forgive this noob question...
>
> I see there are 2 error_count columns (one for each kind of worker)
> but I was wondering why it is useful for users to be able to
> distinguish if the error came from the tablesync workers or from the
> apply workers? Do you have any example?
>
> Also, IIRC sometimes the tablesync might actually do a few "apply"
> changes itself... so the distinction may become a bit fuzzy...
I think that the tablesync phase and the apply phase can fail for
different reasons. So these values would be a good indicator for users
to check if each phase works fine.
After more thoughts, I think it's better to increment sync_error_count
also when a tablesync worker fails while applying the changes. These
counters will correspond to the error information entries that will be
stored in a system catalog.
>
> ~~~
>
> 7. src/backend/postmaster/pgstat.c - comment
>
> @@ -1313,13 +1312,13 @@ pgstat_vacuum_stat(void)
> }
>
> /*
> - * Repeat for subscription workers. Similarly, we needn't bother in the
> - * common case where no subscription workers' stats are being collected.
> + * Repeat for subscription. Similarly, we needn't bother in the common
> + * case where no subscription stats are being collected.
> */
>
> typo?
>
> "Repeat for subscription." --> "Repeat for subscriptions."
Fixed.
>
> ~~~
>
> 8. src/backend/postmaster/pgstat.c
>
> @@ -3000,32 +2968,29 @@ pgstat_fetch_stat_funcentry(Oid func_id)
>
> /*
> * ---------
> - * pgstat_fetch_stat_subworker_entry() -
> + * pgstat_fetch_stat_subentry() -
> *
> * Support function for the SQL-callable pgstat* functions. Returns
> - * the collected statistics for subscription worker or NULL.
> + * the collected statistics for subscription or NULL.
> * ---------
> */
> -PgStat_StatSubWorkerEntry *
> -pgstat_fetch_stat_subworker_entry(Oid subid, Oid subrelid)
> +PgStat_StatSubEntry *
> +pgstat_fetch_stat_subentry(Oid subid)
>
> There seems some kind of inconsistency because the member name is
> called "subscriptions" but sometimes it seems singular.
>
> Some places (e.g. pgstat_vacuum_stat) will iterate multiple results,
> but then other places (like this function) just return to a single
> "subscription" (or "entry").
>
> I suspect all the code may be fine; probably it is just some
> inconsistent (singular/plural) comments that have confused things a
> bit.
Fixed.
>
> ~~~
>
> 9. src/backend/replication/logical/worker.c - subscription id
>
> + /* Report the worker failed during table synchronization */
> + pgstat_report_subscription_error(MyLogicalRepWorker->subid, false);
>
> and
>
> + /* Report the worker failed during the application of the change */
> + pgstat_report_subscription_error(MyLogicalRepWorker->subid, true);
>
>
> Why don't these use MySubscription->oid instead of MyLogicalRepWorker->subid?
It's just because we used to use MyLogicalRepWorker->subid, is there
any particular reason why we should use MySubscription->oid here?
>
> ~~~
>
> 10. src/include/pgstat.h - enum order
>
> @@ -84,8 +84,8 @@ typedef enum StatMsgType
> PGSTAT_MTYPE_REPLSLOT,
> PGSTAT_MTYPE_CONNECT,
> PGSTAT_MTYPE_DISCONNECT,
> + PGSTAT_MTYPE_SUBSCRIPTIONERROR,
> PGSTAT_MTYPE_SUBSCRIPTIONPURGE,
> - PGSTAT_MTYPE_SUBWORKERERROR,
> } StatMsgType;
>
> This change rearranges the enum order. Maybe it is safer not to do this?
>
I assume you're concerned about binary compatibility or something. I
think it should not be a problem since both
PGSTAT_MTYPE_SUBWORKERERROR and PGSTAT_MTYPE_SUBSCRIPTIONPURGE are
introduced to PG15.
> ~~~
>
> 11. src/include/pgstat.h
>
> @@ -767,8 +747,8 @@ typedef union PgStat_Msg
> PgStat_MsgReplSlot msg_replslot;
> PgStat_MsgConnect msg_connect;
> PgStat_MsgDisconnect msg_disconnect;
> + PgStat_MsgSubscriptionError msg_subscriptionerror;
> PgStat_MsgSubscriptionPurge msg_subscriptionpurge;
> - PgStat_MsgSubWorkerError msg_subworkererror;
> } PgStat_Msg;
>
> This change also rearranges the order. Maybe there was no good reason
> to do that?
It's for keeping the alphabetical order within subscription-related messages.
>
> ~~~
>
> 12. src/include/pgstat.h - PgStat_StatDBEntry
>
> @@ -823,16 +803,12 @@ typedef struct PgStat_StatDBEntry
> TimestampTz stats_timestamp; /* time of db stats file update */
>
> /*
> - * tables, functions, and subscription workers must be last in the struct,
> - * because we don't write the pointers out to the stats file.
> - *
> - * subworkers is the hash table of PgStat_StatSubWorkerEntry which stores
> - * statistics of logical replication workers: apply worker and table sync
> - * worker.
> + * tables, functions, and subscription must be last in the struct, because
> + * we don't write the pointers out to the stats file.
> */
>
> Should that say "tables, functions, and subscriptions" (plural)
This part is removed in the latest patch.
On Wed, Feb 23, 2022 at 12:00 PM [email protected]
<[email protected]> wrote:
>
>
> Thanks for your patch.
>
> Few comments:
>
> 1.
> + <structfield>apply_error_count</structfield> <type>uint8</type>
> ...
> + <structfield>sync_error_count</structfield> <type>uint8</type>
>
> It seems that Postgres has no data type named uint8, should we change it to
> bigint?
Right, fixed.
>
> 2.
> +# Wait for the table sync error to be reported.
> +$node_subscriber->poll_query_until(
> + 'postgres',
> + qq[
> +SELECT apply_error_count = 0 AND sync_error_count > 0
> +FROM pg_stat_subscription_activity
> +WHERE subname = 'tap_sub'
> +]) or die "Timed out while waiting for table sync error";
>
> We want to check table sync error here, but do we need to check
> "apply_error_count = 0"? I am not sure if it is possible that the apply worker has
> an unexpected error, which would cause this test to fail.
Yeah, it seems better not to have this condition, fixed.
On Wed, Feb 23, 2022 at 3:21 PM Peter Smith <[email protected]> wrote:
>
> Below are my review comments just for the v1 patch test code.
>
> ======
> 1. "table sync error" versus "tablesync error"
>
> +# Wait for the table sync error to be reported.
> +$node_subscriber->poll_query_until(
> + 'postgres',
> + qq[
> +SELECT apply_error_count = 0 AND sync_error_count > 0
> +FROM pg_stat_subscription_activity
> +WHERE subname = 'tap_sub'
> +]) or die "Timed out while waiting for table sync error";
> +
> +# Truncate test_tab1 so that table sync can continue.
> +$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
> +
> # Wait for initial table sync for test_tab1 to finish.
>
> IMO all these "table sync" should be changed to "tablesync", because a
> table "sync error" sounds like something completely different to a
> "tablesync error".
>
> SUGGESTIONS
> - "Wait for the table sync error to be reported." --> "Wait for the
> tablesync error to be reported."
> - "Timed out while waiting for table sync error" --> "Timed out while
> waiting for tablesync error"
> - "Truncate test_tab1 so that table sync can continue." --> "Truncate
> test_tab1 so that tablesync worker can fun to completion."
> - "Wait for initial table sync for test_tab1 to finish." --> "Wait for
> the tablesync worker of test_tab1 to finish."
Fixed.
>
> ~~~
>
> 3. Wait for the apply worker error
>
> +# Wait for the apply error to be reported.
> +$node_subscriber->poll_query_until(
> + 'postgres',
> + qq[
> +SELECT apply_error_count > 0 AND sync_error_count > 0
> +FROM pg_stat_subscription_activity
> +WHERE subname = 'tap_sub'
> +]) or die "Timed out while waiting for apply error";
>
> This test is only for apply worker errors. So why is the test SQL
> checking "AND sync_error_count > 0"?
>
> (This is similar to what [Tang] #2 reported, but I think she was
> referring to the other tablesync test)
Fixed.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
Attachments:
[application/octet-stream] v2-0001-Reconsider-pg_stat_subscription_workers-view.patch (71.5K, ../../CAD21AoCCaQfGzDCAJgYH=yo72GXozpoMF4fau=5gViXK+uquWw@mail.gmail.com/2-v2-0001-Reconsider-pg_stat_subscription_workers-view.patch)
download | inline diff:
From cef5caf1c89b6e2ce5a67591627e67081591d4c9 Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <[email protected]>
Date: Tue, 22 Feb 2022 10:21:53 +0900
Subject: [PATCH v2] Reconsider pg_stat_subscription_workers view.
It was decided (refer to the Discussion link below) that the stats
collector is not an appropriate place to store the error information of
subscription workers.
This patch changes the pg_stat_subscription_workers view (introduced
by commit 8d74fc9) so that it stores only statistics counters:
apply_error_count and sync_error_count, and has one entry for
subscription. The removed error information such as error-XID and the
error message would be stored in another way in the future in which is
more reliable and persistent.
After removing these error details, there are no longer relation
information, so the subscription statistics are now a cluster-wide
statistics.
The patch also changes the view name to pg_stat_subscription_activity
since the word "worker" is an implementation detail that we use one
worker for one tablesync and one apply.
Discussion: https://postgr.es/m/[email protected]
---
doc/src/sgml/logical-replication.sgml | 4 +-
doc/src/sgml/monitoring.sgml | 104 +---
src/backend/catalog/system_functions.sql | 4 +-
src/backend/catalog/system_views.sql | 27 +-
src/backend/postmaster/pgstat.c | 583 +++++++++-----------
src/backend/replication/logical/worker.c | 45 +-
src/backend/utils/adt/pgstatfuncs.c | 156 ++----
src/include/catalog/pg_proc.dat | 27 +-
src/include/pgstat.h | 116 ++--
src/test/regress/expected/rules.out | 23 +-
src/test/subscription/t/026_stats.pl | 102 ++++
src/test/subscription/t/026_worker_stats.pl | 165 ------
src/tools/pgindent/typedefs.list | 5 +-
13 files changed, 537 insertions(+), 824 deletions(-)
create mode 100644 src/test/subscription/t/026_stats.pl
delete mode 100644 src/test/subscription/t/026_worker_stats.pl
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 96b4886e08..fb4472356d 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -346,9 +346,7 @@
<para>
A conflict will produce an error and will stop the replication; it must be
resolved manually by the user. Details about the conflict can be found in
- <link linkend="monitoring-pg-stat-subscription-workers">
- <structname>pg_stat_subscription_workers</structname></link> and the
- subscriber's server log.
+ the subscriber's server log.
</para>
<para>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index bf7625d988..59294f87a4 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -628,11 +628,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</row>
<row>
- <entry><structname>pg_stat_subscription_workers</structname><indexterm><primary>pg_stat_subscription_workers</primary></indexterm></entry>
- <entry>One row per subscription worker, showing statistics about errors
- that occurred on that subscription worker.
- See <link linkend="monitoring-pg-stat-subscription-workers">
- <structname>pg_stat_subscription_workers</structname></link> for details.
+ <entry><structname>pg_stat_subscription_activity</structname><indexterm><primary>pg_stat_subscription_activity</primary></indexterm></entry>
+ <entry>One row per subscription, showing statistics about subscription
+ activity.
+ See <link linkend="monitoring-pg-stat-subscription-activity">
+ <structname>pg_stat_subscription_activity</structname></link> for details.
</entry>
</row>
@@ -3063,23 +3063,20 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
</sect2>
- <sect2 id="monitoring-pg-stat-subscription-workers">
- <title><structname>pg_stat_subscription_workers</structname></title>
+ <sect2 id="monitoring-pg-stat-subscription-activity">
+ <title><structname>pg_stat_subscription_activity</structname></title>
<indexterm>
- <primary>pg_stat_subscription_workers</primary>
+ <primary>pg_stat_subscription_activity</primary>
</indexterm>
<para>
- The <structname>pg_stat_subscription_workers</structname> view will contain
- one row per subscription worker on which errors have occurred, for workers
- applying logical replication changes and workers handling the initial data
- copy of the subscribed tables. The statistics entry is removed when the
- corresponding subscription is dropped.
+ The <structname>pg_stat_subscription_activity</structname> view will contain
+ one row per subscription.
</para>
- <table id="pg-stat-subscription-workers" xreflabel="pg_stat_subscription_workers">
- <title><structname>pg_stat_subscription_workers</structname> View</title>
+ <table id="pg-stat-subscription-activity" xreflabel="pg_stat_subscription_activity">
+ <title><structname>pg_stat_subscription_activity</structname> View</title>
<tgroup cols="1">
<thead>
<row>
@@ -3113,69 +3110,20 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>subrelid</structfield> <type>oid</type>
+ <structfield>apply_error_count</structfield> <type>bigint</type>
</para>
<para>
- OID of the relation that the worker is synchronizing; null for the
- main apply worker
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_relid</structfield> <type>oid</type>
- </para>
- <para>
- OID of the relation that the worker was processing when the
- error occurred
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_command</structfield> <type>text</type>
- </para>
- <para>
- Name of command being applied when the error occurred. This field
- is null if the error was reported during the initial data copy.
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_xid</structfield> <type>xid</type>
- </para>
- <para>
- Transaction ID of the publisher node being applied when the error
- occurred. This field is null if the error was reported
- during the initial data copy.
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_count</structfield> <type>uint8</type>
- </para>
- <para>
- Number of consecutive times the error occurred
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_message</structfield> <type>text</type>
- </para>
- <para>
- The error message
+ Number of times the error occurred during the application of changes
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_time</structfield> <type>timestamp with time zone</type>
+ <structfield>sync_error_count</structfield> <type>bigint</type>
</para>
<para>
- Last time at which this error occurred
+ Number of times the error occurred during the initial table
+ synchronization
</para></entry>
</row>
@@ -5320,22 +5268,16 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
- <primary>pg_stat_reset_subscription_worker</primary>
+ <primary>pg_stat_reset_subscription_activity</primary>
</indexterm>
- <function>pg_stat_reset_subscription_worker</function> ( <parameter>subid</parameter> <type>oid</type> <optional>, <parameter>relid</parameter> <type>oid</type> </optional> )
+ <function>pg_stat_reset_subscription_activity</function> ( <type>oid</type> )
<returnvalue>void</returnvalue>
</para>
<para>
- Resets the statistics of subscription workers running on the
- subscription with <parameter>subid</parameter> shown in the
- <structname>pg_stat_subscription_workers</structname> view. If the
- argument <parameter>relid</parameter> is not <literal>NULL</literal>,
- resets statistics of the subscription worker handling the initial data
- copy of the relation with <parameter>relid</parameter>. Otherwise,
- resets the subscription worker statistics of the main apply worker.
- If the argument <parameter>relid</parameter> is omitted, resets the
- statistics of all subscription workers running on the subscription
- with <parameter>subid</parameter>.
+ Resets statistics for a single subscription shown in the
+ <structname>pg_stat_subscription_activity</structname> view to zero. If
+ the argument is <literal>NULL</literal>, reset statistics for all
+ subscriptions.
</para>
<para>
This function is restricted to superusers by default, but other users
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index fd1421788e..a2f6b15d2a 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -639,9 +639,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_reset_single_function_counters(oid) FROM publ
REVOKE EXECUTE ON FUNCTION pg_stat_reset_replication_slot(text) FROM public;
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid) FROM public;
-
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid, oid) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_activity(oid) FROM public;
REVOKE EXECUTE ON FUNCTION lo_import(text) FROM public;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 3cb69b1f87..db56e84c1f 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1264,25 +1264,12 @@ GRANT SELECT (oid, subdbid, subname, subowner, subenabled, subbinary,
substream, subtwophasestate, subslotname, subsynccommit, subpublications)
ON pg_subscription TO public;
-CREATE VIEW pg_stat_subscription_workers AS
+CREATE VIEW pg_stat_subscription_activity AS
SELECT
- w.subid,
+ a.subid,
s.subname,
- w.subrelid,
- w.last_error_relid,
- w.last_error_command,
- w.last_error_xid,
- w.last_error_count,
- w.last_error_message,
- w.last_error_time
- FROM (SELECT
- oid as subid,
- NULL as relid
- FROM pg_subscription
- UNION ALL
- SELECT
- srsubid as subid,
- srrelid as relid
- FROM pg_subscription_rel) sr,
- LATERAL pg_stat_get_subscription_worker(sr.subid, sr.relid) w
- JOIN pg_subscription s ON (w.subid = s.oid);
+ a.apply_error_count,
+ a.sync_error_count,
+ a.stats_reset
+ FROM pg_subscription as s,
+ pg_stat_get_subscription_activity(oid) as a;
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 0646f53098..840b1e38d5 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -106,7 +106,7 @@
#define PGSTAT_DB_HASH_SIZE 16
#define PGSTAT_TAB_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
-#define PGSTAT_SUBWORKER_HASH_SIZE 32
+#define PGSTAT_SUBSCRIPTION_HASH_SIZE 32
#define PGSTAT_REPLSLOT_HASH_SIZE 32
@@ -284,6 +284,7 @@ static PgStat_GlobalStats globalStats;
static PgStat_WalStats walStats;
static PgStat_SLRUStats slruStats[SLRU_NUM_ELEMENTS];
static HTAB *replSlotStatHash = NULL;
+static HTAB *subscriptionStatHash = NULL;
/*
* List of OIDs of databases we need to write out. If an entry is InvalidOid,
@@ -322,14 +323,13 @@ NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_no
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
Oid tableoid, bool create);
-static PgStat_StatSubWorkerEntry *pgstat_get_subworker_entry(PgStat_StatDBEntry *dbentry,
- Oid subid, Oid subrelid,
- bool create);
+static PgStat_StatSubEntry *pgstat_get_subscription_entry(Oid subid, bool create);
+static void pgstat_reset_subscription(PgStat_StatSubEntry *subentry, TimestampTz ts);
static void pgstat_write_statsfiles(bool permanent, bool allDbs);
static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- HTAB *subworkerhash, bool permanent);
+ bool permanent);
static void backend_read_statsfile(void);
static bool pgstat_write_statsfile_needed(void);
@@ -341,7 +341,6 @@ static void pgstat_reset_replslot(PgStat_StatReplSlotEntry *slotstats, Timestamp
static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg, TimestampTz now);
static void pgstat_send_funcstats(void);
static void pgstat_send_slru(void);
-static void pgstat_send_subscription_purge(PgStat_MsgSubscriptionPurge *msg);
static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
static bool pgstat_should_report_connstat(void);
static void pgstat_report_disconnect(Oid dboid);
@@ -363,6 +362,7 @@ static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, in
static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
static void pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len);
static void pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg, int len);
+static void pgstat_recv_resetsubcounter(PgStat_MsgResetsubcounter *msg, int len);
static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
@@ -381,7 +381,7 @@ static void pgstat_recv_disconnect(PgStat_MsgDisconnect *msg, int len);
static void pgstat_recv_replslot(PgStat_MsgReplSlot *msg, int len);
static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
static void pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len);
-static void pgstat_recv_subworker_error(PgStat_MsgSubWorkerError *msg, int len);
+static void pgstat_recv_subscription_error(PgStat_MsgSubscriptionError *msg, int len);
/* ------------------------------------------------------------
* Public functions called from postmaster follow
@@ -1187,6 +1187,32 @@ pgstat_vacuum_stat(void)
}
}
+ /*
+ * Repeat the above steps for subscriptions, if subscription stats are
+ * being collected.
+ */
+ if (subscriptionStatHash)
+ {
+ PgStat_StatSubEntry *subentry;
+
+ /*
+ * Read pg_subscription and make a list of OIDs of all existing
+ * subscriptions
+ */
+ htab = pgstat_collect_oids(SubscriptionRelationId, Anum_pg_subscription_oid);
+
+ hash_seq_init(&hstat, subscriptionStatHash);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&hstat)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (hash_search(htab, (void *) &(subentry->subid), HASH_FIND, NULL) == NULL)
+ pgstat_report_subscription_drop(subentry->subid);
+ }
+
+ hash_destroy(htab);
+ }
+
/*
* Lookup our own database entry; if not found, nothing more to do.
*/
@@ -1311,74 +1337,6 @@ pgstat_vacuum_stat(void)
hash_destroy(htab);
}
-
- /*
- * Repeat for subscription workers. Similarly, we needn't bother in the
- * common case where no subscription workers' stats are being collected.
- */
- if (dbentry->subworkers != NULL &&
- hash_get_num_entries(dbentry->subworkers) > 0)
- {
- PgStat_StatSubWorkerEntry *subwentry;
- PgStat_MsgSubscriptionPurge spmsg;
-
- /*
- * Read pg_subscription and make a list of OIDs of all existing
- * subscriptions
- */
- htab = pgstat_collect_oids(SubscriptionRelationId, Anum_pg_subscription_oid);
-
- spmsg.m_databaseid = MyDatabaseId;
- spmsg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&hstat)) != NULL)
- {
- bool exists = false;
- Oid subid = subwentry->key.subid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &subid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * It is possible that we have multiple entries for the
- * subscription corresponding to apply worker and tablesync
- * workers. In such cases, we don't need to add the same subid
- * again.
- */
- for (int i = 0; i < spmsg.m_nentries; i++)
- {
- if (spmsg.m_subids[i] == subid)
- {
- exists = true;
- break;
- }
- }
-
- if (exists)
- continue;
-
- /* This subscription is dead, add the subid to the message */
- spmsg.m_subids[spmsg.m_nentries++] = subid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (spmsg.m_nentries >= PGSTAT_NUM_SUBSCRIPTIONPURGE)
- {
- pgstat_send_subscription_purge(&spmsg);
- spmsg.m_nentries = 0;
- }
- }
-
- /* Send the rest of dead subscriptions */
- if (spmsg.m_nentries > 0)
- pgstat_send_subscription_purge(&spmsg);
-
- hash_destroy(htab);
- }
}
@@ -1551,8 +1509,7 @@ pgstat_reset_shared_counters(const char *target)
* ----------
*/
void
-pgstat_reset_single_counter(Oid objoid, Oid subobjoid,
- PgStat_Single_Reset_Type type)
+pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
PgStat_MsgResetsinglecounter msg;
@@ -1563,7 +1520,6 @@ pgstat_reset_single_counter(Oid objoid, Oid subobjoid,
msg.m_databaseid = MyDatabaseId;
msg.m_resettype = type;
msg.m_objectid = objoid;
- msg.m_subobjectid = subobjoid;
pgstat_send(&msg, sizeof(msg));
}
@@ -1623,6 +1579,30 @@ pgstat_reset_replslot_counter(const char *name)
pgstat_send(&msg, sizeof(msg));
}
+/* ----------
+ * pgstat_reset_subscription_counter() -
+ *
+ * Tell the statistics collector to reset a single subscription
+ * counter, or all subscription counters (when subid is InvalidOid).
+ *
+ * Permission checking for this function is managed through the normal
+ * GRANT system.
+ * ----------
+ */
+void
+pgstat_reset_subscription_counter(Oid subid)
+{
+ PgStat_MsgResetsubcounter msg;
+
+ if (pgStatSock == PGINVALID_SOCKET)
+ return;
+
+ msg.m_subid = subid;
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSUBCOUNTER);
+
+ pgstat_send(&msg, sizeof(msg));
+}
+
/* ----------
* pgstat_report_autovac() -
*
@@ -1949,31 +1929,21 @@ pgstat_report_replslot_drop(const char *slotname)
}
/* ----------
- * pgstat_report_subworker_error() -
+ * pgstat_report_subscription_error() -
*
- * Tell the collector about the subscription worker error.
+ * Tell the collector about the subscription error.
* ----------
*/
void
-pgstat_report_subworker_error(Oid subid, Oid subrelid, Oid relid,
- LogicalRepMsgType command, TransactionId xid,
- const char *errmsg)
+pgstat_report_subscription_error(Oid subid, bool is_apply_error)
{
- PgStat_MsgSubWorkerError msg;
- int len;
+ PgStat_MsgSubscriptionError msg;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBWORKERERROR);
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBSCRIPTIONERROR);
msg.m_databaseid = MyDatabaseId;
msg.m_subid = subid;
- msg.m_subrelid = subrelid;
- msg.m_relid = relid;
- msg.m_command = command;
- msg.m_xid = xid;
- msg.m_timestamp = GetCurrentTimestamp();
- strlcpy(msg.m_message, errmsg, PGSTAT_SUBWORKERERROR_MSGLEN);
-
- len = offsetof(PgStat_MsgSubWorkerError, m_message) + strlen(msg.m_message) + 1;
- pgstat_send(&msg, len);
+ msg.m_is_apply_error = is_apply_error;
+ pgstat_send(&msg, sizeof(PgStat_MsgSubscriptionError));
}
/* ----------
@@ -1987,10 +1957,9 @@ pgstat_report_subscription_drop(Oid subid)
{
PgStat_MsgSubscriptionPurge msg;
- msg.m_databaseid = MyDatabaseId;
- msg.m_subids[0] = subid;
- msg.m_nentries = 1;
- pgstat_send_subscription_purge(&msg);
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBSCRIPTIONPURGE);
+ msg.m_subid = subid;
+ pgstat_send(&msg, sizeof(PgStat_MsgSubscriptionPurge));
}
/* ----------
@@ -3000,32 +2969,19 @@ pgstat_fetch_stat_funcentry(Oid func_id)
/*
* ---------
- * pgstat_fetch_stat_subworker_entry() -
+ * pgstat_fetch_stat_subscription() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for subscription worker or NULL.
+ * the collected statistics for one subscription or NULL.
* ---------
*/
-PgStat_StatSubWorkerEntry *
-pgstat_fetch_stat_subworker_entry(Oid subid, Oid subrelid)
+PgStat_StatSubEntry *
+pgstat_fetch_stat_subscription(Oid subid)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *wentry = NULL;
-
/* Load the stats file if needed */
backend_read_statsfile();
- /*
- * Lookup our database, then find the requested subscription worker stats.
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->subworkers != NULL)
- {
- wentry = pgstat_get_subworker_entry(dbentry, subid, subrelid,
- false);
- }
-
- return wentry;
+ return pgstat_get_subscription_entry(subid, false);
}
/*
@@ -3465,24 +3421,6 @@ pgstat_send_slru(void)
}
}
-/* --------
- * pgstat_send_subscription_purge() -
- *
- * Send a subscription purge message to the collector
- * --------
- */
-static void
-pgstat_send_subscription_purge(PgStat_MsgSubscriptionPurge *msg)
-{
- int len;
-
- len = offsetof(PgStat_MsgSubscriptionPurge, m_subids[0])
- + msg->m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg->m_hdr, PGSTAT_MTYPE_SUBSCRIPTIONPURGE);
- pgstat_send(msg, len);
-}
-
/* ----------
* PgstatCollectorMain() -
*
@@ -3668,6 +3606,10 @@ PgstatCollectorMain(int argc, char *argv[])
len);
break;
+ case PGSTAT_MTYPE_RESETSUBCOUNTER:
+ pgstat_recv_resetsubcounter(&msg.msg_resetsubcounter, len);
+ break;
+
case PGSTAT_MTYPE_AUTOVAC_START:
pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
break;
@@ -3742,8 +3684,8 @@ PgstatCollectorMain(int argc, char *argv[])
pgstat_recv_subscription_purge(&msg.msg_subscriptionpurge, len);
break;
- case PGSTAT_MTYPE_SUBWORKERERROR:
- pgstat_recv_subworker_error(&msg.msg_subworkererror, len);
+ case PGSTAT_MTYPE_SUBSCRIPTIONERROR:
+ pgstat_recv_subscription_error(&msg.msg_subscriptionerror, len);
break;
default:
@@ -3791,8 +3733,7 @@ PgstatCollectorMain(int argc, char *argv[])
/*
* Subroutine to clear stats in a database entry
*
- * Tables, functions, and subscription workers hashes are initialized
- * to empty.
+ * Tables and functions hashes are initialized to empty.
*/
static void
reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
@@ -3845,13 +3786,6 @@ reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
PGSTAT_FUNCTION_HASH_SIZE,
&hash_ctl,
HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(PgStat_StatSubWorkerKey);
- hash_ctl.entrysize = sizeof(PgStat_StatSubWorkerEntry);
- dbentry->subworkers = hash_create("Per-database subscription worker",
- PGSTAT_SUBWORKER_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
}
/*
@@ -3876,7 +3810,7 @@ pgstat_get_db_entry(Oid databaseid, bool create)
/*
* If not found, initialize the new one. This creates empty hash tables
- * for tables, functions, and subscription workers, too.
+ * for tables and functions, too.
*/
if (!found)
reset_dbentry_counters(result);
@@ -3935,45 +3869,65 @@ pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
}
/* ----------
- * pgstat_get_subworker_entry
+ * pgstat_get_subscription_entry
*
- * Return subscription worker entry with the given subscription OID and
- * relation OID. If subrelid is InvalidOid, it returns an entry of the
- * apply worker otherwise returns an entry of the table sync worker
- * associated with subrelid. If no subscription worker entry exists,
- * initialize it, if the create parameter is true. Else, return NULL.
+ * Return the subscription statistics entry with the given subscription OID.
+ * If no subscription entry exists, initialize it, if the create parameter is
+ * true. Else, return NULL.
* ----------
*/
-static PgStat_StatSubWorkerEntry *
-pgstat_get_subworker_entry(PgStat_StatDBEntry *dbentry, Oid subid, Oid subrelid,
- bool create)
+static PgStat_StatSubEntry *
+pgstat_get_subscription_entry(Oid subid, bool create)
{
- PgStat_StatSubWorkerEntry *subwentry;
- PgStat_StatSubWorkerKey key;
+ PgStat_StatSubEntry *subentry;
bool found;
HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
- key.subid = subid;
- key.subrelid = subrelid;
- subwentry = (PgStat_StatSubWorkerEntry *) hash_search(dbentry->subworkers,
- (void *) &key,
- action, &found);
+ if (subscriptionStatHash == NULL)
+ {
+ HASHCTL hash_ctl;
+
+ /*
+ * Quick return NULL if the hash table is empty and the caller didn't
+ * request to create the entry.
+ */
+ if (!create)
+ return NULL;
+
+ hash_ctl.keysize = sizeof(Oid);
+ hash_ctl.entrysize = sizeof(PgStat_StatSubEntry);
+ subscriptionStatHash = hash_create("Subscription hash",
+ PGSTAT_SUBSCRIPTION_HASH_SIZE,
+ &hash_ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ subentry = (PgStat_StatSubEntry *) hash_search(subscriptionStatHash,
+ (void *) &subid,
+ action, &found);
if (!create && !found)
return NULL;
/* If not found, initialize the new one */
if (!found)
- {
- subwentry->last_error_relid = InvalidOid;
- subwentry->last_error_command = 0;
- subwentry->last_error_xid = InvalidTransactionId;
- subwentry->last_error_count = 0;
- subwentry->last_error_time = 0;
- subwentry->last_error_message[0] = '\0';
- }
+ pgstat_reset_subscription(subentry, 0);
- return subwentry;
+ return subentry;
+}
+
+/* ----------
+ * pgstat_reset_subscription
+ *
+ * Reset the given subscription stats.
+ * ----------
+ */
+static void
+pgstat_reset_subscription(PgStat_StatSubEntry *subentry, TimestampTz ts)
+{
+ subentry->apply_error_count = 0;
+ subentry->sync_error_count = 0;
+ subentry->stat_reset_timestamp = ts;
}
/* ----------
@@ -4059,8 +4013,8 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
{
/*
- * Write out the table, function, and subscription-worker stats for
- * this DB into the appropriate per-DB stat file, if required.
+ * Write out the table and function stats for this DB into the
+ * appropriate per-DB stat file, if required.
*/
if (allDbs || pgstat_db_requested(dbentry->databaseid))
{
@@ -4095,6 +4049,22 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
}
}
+ /*
+ * Write subscription stats struct
+ */
+ if (subscriptionStatHash)
+ {
+ PgStat_StatSubEntry *subentry;
+
+ hash_seq_init(&hstat, subscriptionStatHash);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&hstat)) != NULL)
+ {
+ fputc('S', fpout);
+ rc = fwrite(subentry, sizeof(PgStat_StatSubEntry), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+ }
+
/*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
@@ -4174,10 +4144,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
{
HASH_SEQ_STATUS tstat;
HASH_SEQ_STATUS fstat;
- HASH_SEQ_STATUS sstat;
PgStat_StatTabEntry *tabentry;
PgStat_StatFuncEntry *funcentry;
- PgStat_StatSubWorkerEntry *subwentry;
FILE *fpout;
int32 format_id;
Oid dbid = dbentry->databaseid;
@@ -4232,17 +4200,6 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
(void) rc; /* we'll check for error with ferror */
}
- /*
- * Walk through the database's subscription worker stats table.
- */
- hash_seq_init(&sstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&sstat)) != NULL)
- {
- fputc('S', fpout);
- rc = fwrite(subwentry, sizeof(PgStat_StatSubWorkerEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
/*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
@@ -4301,9 +4258,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
* files after reading; the in-memory status is now authoritative, and the
* files would be out of date in case somebody else reads them.
*
- * If a 'deep' read is requested, table/function/subscription-worker stats are
- * read, otherwise the table/function/subscription-worker hash tables remain
- * empty.
+ * If a 'deep' read is requested, table/function stats are read, otherwise
+ * the table/function hash tables remain empty.
* ----------
*/
static HTAB *
@@ -4482,7 +4438,6 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
dbentry->tables = NULL;
dbentry->functions = NULL;
- dbentry->subworkers = NULL;
/*
* In the collector, disregard the timestamp we read from the
@@ -4494,8 +4449,8 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
dbentry->stats_timestamp = 0;
/*
- * Don't create tables/functions/subworkers hashtables for
- * uninteresting databases.
+ * Don't create tables/functions hashtables for uninteresting
+ * databases.
*/
if (onlydb != InvalidOid)
{
@@ -4520,14 +4475,6 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
&hash_ctl,
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
- hash_ctl.keysize = sizeof(PgStat_StatSubWorkerKey);
- hash_ctl.entrysize = sizeof(PgStat_StatSubWorkerEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->subworkers = hash_create("Per-database subscription worker",
- PGSTAT_SUBWORKER_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
/*
* If requested, read the data from the database-specific
* file. Otherwise we just leave the hashtables empty.
@@ -4536,7 +4483,6 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
pgstat_read_db_statsfile(dbentry->databaseid,
dbentry->tables,
dbentry->functions,
- dbentry->subworkers,
permanent);
break;
@@ -4580,6 +4526,45 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
break;
}
+ /*
+ * 'S' A PgStat_StatSubEntry struct describing subscription
+ * statistics.
+ */
+ case 'S':
+ {
+ PgStat_StatSubEntry subbuf;
+ PgStat_StatSubEntry *subentry;
+
+ if (fread(&subbuf, 1, sizeof(PgStat_StatSubEntry),
+ fpin) != sizeof(PgStat_StatSubEntry))
+ {
+ ereport(pgStatRunningInCollector ? LOG : WARNING,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ if (subscriptionStatHash == NULL)
+ {
+ HASHCTL hash_ctl;
+
+ hash_ctl.keysize = sizeof(Oid);
+ hash_ctl.entrysize = sizeof(PgStat_StatSubEntry);
+ hash_ctl.hcxt = pgStatLocalContext;
+ subscriptionStatHash = hash_create("Subscription hash",
+ PGSTAT_SUBSCRIPTION_HASH_SIZE,
+ &hash_ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
+
+ subentry = (PgStat_StatSubEntry *) hash_search(subscriptionStatHash,
+ (void *) &subbuf.subid,
+ HASH_ENTER, NULL);
+
+ memcpy(subentry, &subbuf, sizeof(subbuf));
+ break;
+ }
+
case 'E':
goto done;
@@ -4614,21 +4599,19 @@ done:
* As in pgstat_read_statsfiles, if the permanent file is requested, it is
* removed after reading.
*
- * Note: this code has the ability to skip storing per-table, per-function, or
- * per-subscription-worker data, if NULL is passed for the corresponding hashtable.
- * That's not used at the moment though.
+ * Note: this code has the ability to skip storing per-table or per-function
+ * data, if NULL is passed for the corresponding hashtable. That's not used
+ * at the moment though.
* ----------
*/
static void
pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- HTAB *subworkerhash, bool permanent)
+ bool permanent)
{
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
- PgStat_StatSubWorkerEntry subwbuf;
- PgStat_StatSubWorkerEntry *subwentry;
FILE *fpin;
int32 format_id;
bool found;
@@ -4742,41 +4725,6 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
memcpy(funcentry, &funcbuf, sizeof(funcbuf));
break;
- /*
- * 'S' A PgStat_StatSubWorkerEntry struct describing
- * subscription worker statistics.
- */
- case 'S':
- if (fread(&subwbuf, 1, sizeof(PgStat_StatSubWorkerEntry),
- fpin) != sizeof(PgStat_StatSubWorkerEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if subscription worker data not wanted.
- */
- if (subworkerhash == NULL)
- break;
-
- subwentry = (PgStat_StatSubWorkerEntry *) hash_search(subworkerhash,
- (void *) &subwbuf.key,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(subwentry, &subwbuf, sizeof(subwbuf));
- break;
-
/*
* 'E' The EOF marker of a complete stats file.
*/
@@ -4829,6 +4777,7 @@ pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
PgStat_WalStats myWalStats;
PgStat_SLRUStats mySLRUStats[SLRU_NUM_ELEMENTS];
PgStat_StatReplSlotEntry myReplSlotStats;
+ PgStat_StatSubEntry mySubStats;
FILE *fpin;
int32 format_id;
const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
@@ -4959,6 +4908,22 @@ pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
}
break;
+ /*
+ * 'S' A PgStat_StatSubEntry struct describing subscription
+ * statistics follows.
+ */
+ case 'S':
+ if (fread(&mySubStats, 1, sizeof(PgStat_StatSubEntry), fpin)
+ != sizeof(PgStat_StatSubEntry))
+ {
+ ereport(pgStatRunningInCollector ? LOG : WARNING,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ FreeFile(fpin);
+ return false;
+ }
+ break;
+
case 'E':
goto done;
@@ -5164,6 +5129,7 @@ pgstat_clear_snapshot(void)
pgStatLocalContext = NULL;
pgStatDBHash = NULL;
replSlotStatHash = NULL;
+ subscriptionStatHash = NULL;
/*
* Historically the backend_status.c facilities lived in this file, and
@@ -5450,8 +5416,6 @@ pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
hash_destroy(dbentry->tables);
if (dbentry->functions != NULL)
hash_destroy(dbentry->functions);
- if (dbentry->subworkers != NULL)
- hash_destroy(dbentry->subworkers);
if (hash_search(pgStatDBHash,
(void *) &dbid,
@@ -5489,16 +5453,13 @@ pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
hash_destroy(dbentry->tables);
if (dbentry->functions != NULL)
hash_destroy(dbentry->functions);
- if (dbentry->subworkers != NULL)
- hash_destroy(dbentry->subworkers);
dbentry->tables = NULL;
dbentry->functions = NULL;
- dbentry->subworkers = NULL;
/*
* Reset database-level stats, too. This creates empty hash tables for
- * tables, functions, and subscription workers.
+ * tables and functions.
*/
reset_dbentry_counters(dbentry);
}
@@ -5567,14 +5528,6 @@ pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
else if (msg->m_resettype == RESET_FUNCTION)
(void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_SUBWORKER)
- {
- PgStat_StatSubWorkerKey key;
-
- key.subid = msg->m_objectid;
- key.subrelid = msg->m_subobjectid;
- (void) hash_search(dbentry->subworkers, (void *) &key, HASH_REMOVE, NULL);
- }
}
/* ----------
@@ -5645,6 +5598,51 @@ pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg,
}
}
+/* ----------
+ * pgstat_recv_resetsubcounter() -
+ *
+ * Reset some subscription statistics of the cluster.
+ * ----------
+ */
+static void
+pgstat_recv_resetsubcounter(PgStat_MsgResetsubcounter *msg, int len)
+{
+ PgStat_StatSubEntry *subentry;
+ TimestampTz ts;
+
+ /* Return if we don't have replication subscription statistics */
+ if (subscriptionStatHash == NULL)
+ return;
+
+ ts = GetCurrentTimestamp();
+ if (!OidIsValid(msg->m_subid))
+ {
+ HASH_SEQ_STATUS sstat;
+
+ /* Clear all subscription counters */
+ hash_seq_init(&sstat, subscriptionStatHash);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&sstat)) != NULL)
+ pgstat_reset_subscription(subentry, ts);
+ }
+ else
+ {
+ /* Get the subscription statistics to reset */
+ subentry = pgstat_get_subscription_entry(msg->m_subid, false);
+
+ /*
+ * Nothing to do if the given subscription entry is not found. This
+ * could happen when the subscription with the subid is removed and
+ * the corresponding statistics entry is also removed before receiving
+ * the reset message.
+ */
+ if (!subentry)
+ return;
+
+ /* Reset the stats for the requested replication slot */
+ pgstat_reset_subscription(subentry, ts);
+ }
+}
+
/* ----------
* pgstat_recv_autovac() -
@@ -6126,73 +6124,34 @@ pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
static void
pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *subwentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /* No need to purge if we don't even know the database */
- if (!dbentry || !dbentry->subworkers)
+ /* Return if we don't have replication subscription statistics */
+ if (subscriptionStatHash == NULL)
return;
- /* Remove all subscription worker statistics for the given subscriptions */
- hash_seq_init(&hstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&hstat)) != NULL)
- {
- for (int i = 0; i < msg->m_nentries; i++)
- {
- if (subwentry->key.subid == msg->m_subids[i])
- {
- (void) hash_search(dbentry->subworkers, (void *) &(subwentry->key),
- HASH_REMOVE, NULL);
- break;
- }
- }
- }
+ /* Remove from hashtable if present; we don't care if it's not */
+ (void) hash_search(subscriptionStatHash, (void *) &(msg->m_subid),
+ HASH_REMOVE, NULL);
}
/* ----------
- * pgstat_recv_subworker_error() -
+ * pgstat_recv_subscription_error() -
*
- * Process a SUBWORKERERROR message.
+ * Process a SUBSCRIPTIONERROR message.
* ----------
*/
static void
-pgstat_recv_subworker_error(PgStat_MsgSubWorkerError *msg, int len)
+pgstat_recv_subscription_error(PgStat_MsgSubscriptionError *msg, int len)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *subwentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
+ PgStat_StatSubEntry *subentry;
- /* Get the subscription worker stats */
- subwentry = pgstat_get_subworker_entry(dbentry, msg->m_subid,
- msg->m_subrelid, true);
- Assert(subwentry);
+ /* Get the subscription stats */
+ subentry = pgstat_get_subscription_entry(msg->m_subid, true);
+ Assert(subentry);
- if (subwentry->last_error_relid == msg->m_relid &&
- subwentry->last_error_command == msg->m_command &&
- subwentry->last_error_xid == msg->m_xid &&
- strcmp(subwentry->last_error_message, msg->m_message) == 0)
- {
- /*
- * The same error occurred again in succession, just update its
- * timestamp and count.
- */
- subwentry->last_error_count++;
- subwentry->last_error_time = msg->m_timestamp;
- return;
- }
-
- /* Otherwise, update the error information */
- subwentry->last_error_relid = msg->m_relid;
- subwentry->last_error_command = msg->m_command;
- subwentry->last_error_xid = msg->m_xid;
- subwentry->last_error_count = 1;
- subwentry->last_error_time = msg->m_timestamp;
- strlcpy(subwentry->last_error_message, msg->m_message,
- PGSTAT_SUBWORKERERROR_MSGLEN);
+ if (msg->m_is_apply_error)
+ subentry->apply_error_count++;
+ else
+ subentry->sync_error_count++;
}
/* ----------
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 5d9acc6173..cf287fd4ff 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -3377,7 +3377,6 @@ void
ApplyWorkerMain(Datum main_arg)
{
int worker_slot = DatumGetInt32(main_arg);
- MemoryContext cctx = CurrentMemoryContext;
MemoryContext oldctx;
char originname[NAMEDATALEN];
XLogRecPtr origin_startpos;
@@ -3485,20 +3484,15 @@ ApplyWorkerMain(Datum main_arg)
}
PG_CATCH();
{
- MemoryContext ecxt = MemoryContextSwitchTo(cctx);
- ErrorData *errdata = CopyErrorData();
-
/*
- * Report the table sync error. There is no corresponding message
- * type for table synchronization.
+ * Abort the current transaction so that we send the stats message
+ * in an idle state.
*/
- pgstat_report_subworker_error(MyLogicalRepWorker->subid,
- MyLogicalRepWorker->relid,
- MyLogicalRepWorker->relid,
- 0, /* message type */
- InvalidTransactionId,
- errdata->message);
- MemoryContextSwitchTo(ecxt);
+ AbortOutOfAnyTransaction();
+
+ /* Report the worker failed during table synchronization */
+ pgstat_report_subscription_error(MyLogicalRepWorker->subid, false);
+
PG_RE_THROW();
}
PG_END_TRY();
@@ -3625,22 +3619,15 @@ ApplyWorkerMain(Datum main_arg)
}
PG_CATCH();
{
- /* report the apply error */
- if (apply_error_callback_arg.command != 0)
- {
- MemoryContext ecxt = MemoryContextSwitchTo(cctx);
- ErrorData *errdata = CopyErrorData();
-
- pgstat_report_subworker_error(MyLogicalRepWorker->subid,
- MyLogicalRepWorker->relid,
- apply_error_callback_arg.rel != NULL
- ? apply_error_callback_arg.rel->localreloid
- : InvalidOid,
- apply_error_callback_arg.command,
- apply_error_callback_arg.remote_xid,
- errdata->message);
- MemoryContextSwitchTo(ecxt);
- }
+ /*
+ * Abort the current transaction so that we send the stats message in
+ * an idle state.
+ */
+ AbortOutOfAnyTransaction();
+
+ /* Report the worker failed during the application of the change */
+ pgstat_report_subscription_error(MyLogicalRepWorker->subid,
+ !am_tablesync_worker());
PG_RE_THROW();
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 30e8dfa7c1..a430ff3da6 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2163,7 +2163,7 @@ pg_stat_reset_single_table_counters(PG_FUNCTION_ARGS)
{
Oid taboid = PG_GETARG_OID(0);
- pgstat_reset_single_counter(taboid, InvalidOid, RESET_TABLE);
+ pgstat_reset_single_counter(taboid, RESET_TABLE);
PG_RETURN_VOID();
}
@@ -2173,38 +2173,11 @@ pg_stat_reset_single_function_counters(PG_FUNCTION_ARGS)
{
Oid funcoid = PG_GETARG_OID(0);
- pgstat_reset_single_counter(funcoid, InvalidOid, RESET_FUNCTION);
+ pgstat_reset_single_counter(funcoid, RESET_FUNCTION);
PG_RETURN_VOID();
}
-Datum
-pg_stat_reset_subscription_worker_subrel(PG_FUNCTION_ARGS)
-{
- Oid subid = PG_GETARG_OID(0);
- Oid relid = PG_ARGISNULL(1) ? InvalidOid : PG_GETARG_OID(1);
-
- pgstat_reset_single_counter(subid, relid, RESET_SUBWORKER);
-
- PG_RETURN_VOID();
-}
-
-/* Reset all subscription worker stats associated with the given subscription */
-Datum
-pg_stat_reset_subscription_worker_sub(PG_FUNCTION_ARGS)
-{
- Oid subid = PG_GETARG_OID(0);
-
- /*
- * Use subscription drop message to remove statistics of all subscription
- * workers.
- */
- pgstat_report_subscription_drop(subid);
-
- PG_RETURN_VOID();
-}
-
-
/* Reset SLRU counters (a specific one or all of them). */
Datum
pg_stat_reset_slru(PG_FUNCTION_ARGS)
@@ -2258,6 +2231,32 @@ pg_stat_reset_replication_slot(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+/* Reset subscription stats (a specific one or all of them) */
+Datum
+pg_stat_reset_subscription_activity(PG_FUNCTION_ARGS)
+{
+ Oid subid;
+
+ if (PG_ARGISNULL(0))
+ {
+ /* Clear all subscription stats */
+ subid = InvalidOid;
+ }
+ else
+ {
+ subid = PG_GETARG_OID(0);
+
+ if (!OidIsValid(subid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid subscription OID %u", subid)));
+ }
+
+ pgstat_reset_subscription_counter(subid);
+
+ PG_RETURN_VOID();
+}
+
Datum
pg_stat_get_archiver(PG_FUNCTION_ARGS)
{
@@ -2400,50 +2399,32 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
}
/*
- * Get the subscription worker statistics for the given subscription
- * (and relation).
+ * Get the subscription activity statistics for the given subscription. If the
+ * subscription statistics is not available, return all-zeros stats.
*/
Datum
-pg_stat_get_subscription_worker(PG_FUNCTION_ARGS)
+pg_stat_get_subscription_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_SUBSCRIPTION_WORKER_COLS 8
+#define PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS 4
Oid subid = PG_GETARG_OID(0);
- Oid subrelid;
TupleDesc tupdesc;
- Datum values[PG_STAT_GET_SUBSCRIPTION_WORKER_COLS];
- bool nulls[PG_STAT_GET_SUBSCRIPTION_WORKER_COLS];
- PgStat_StatSubWorkerEntry *wentry;
- int i;
-
- if (PG_ARGISNULL(1))
- subrelid = InvalidOid;
- else
- subrelid = PG_GETARG_OID(1);
+ Datum values[PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS];
+ bool nulls[PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS];
+ PgStat_StatSubEntry *subentry;
+ PgStat_StatSubEntry allzero;
- /* Get subscription worker stats */
- wentry = pgstat_fetch_stat_subworker_entry(subid, subrelid);
-
- /* Return NULL if there is no worker statistics */
- if (wentry == NULL)
- PG_RETURN_NULL();
+ /* Get subscription stats */
+ subentry = pgstat_fetch_stat_subscription(subid);
/* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_WORKER_COLS);
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "subid",
OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subrelid",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 3, "last_error_relid",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 4, "last_error_command",
- TEXTOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 5, "last_error_xid",
- XIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 6, "last_error_count",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "apply_error_count",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 7, "last_error_message",
- TEXTOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 8, "last_error_time",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "sync_error_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
@@ -2451,46 +2432,27 @@ pg_stat_get_subscription_worker(PG_FUNCTION_ARGS)
MemSet(values, 0, sizeof(values));
MemSet(nulls, 0, sizeof(nulls));
- i = 0;
- /* subid */
- values[i++] = ObjectIdGetDatum(subid);
-
- /* subrelid */
- if (OidIsValid(subrelid))
- values[i++] = ObjectIdGetDatum(subrelid);
- else
- nulls[i++] = true;
-
- /* last_error_relid */
- if (OidIsValid(wentry->last_error_relid))
- values[i++] = ObjectIdGetDatum(wentry->last_error_relid);
- else
- nulls[i++] = true;
-
- /* last_error_command */
- if (wentry->last_error_command != 0)
- values[i++] =
- CStringGetTextDatum(logicalrep_message_type(wentry->last_error_command));
- else
- nulls[i++] = true;
+ if (!subentry)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(PgStat_StatSubEntry));
+ subentry = &allzero;
+ }
- /* last_error_xid */
- if (TransactionIdIsValid(wentry->last_error_xid))
- values[i++] = TransactionIdGetDatum(wentry->last_error_xid);
- else
- nulls[i++] = true;
+ /* subid */
+ values[0] = ObjectIdGetDatum(subid);
- /* last_error_count */
- values[i++] = Int64GetDatum(wentry->last_error_count);
+ /* apply_error_count */
+ values[1] = Int64GetDatum(subentry->apply_error_count);
- /* last_error_message */
- values[i++] = CStringGetTextDatum(wentry->last_error_message);
+ /* sync_error_count */
+ values[2] = Int64GetDatum(subentry->sync_error_count);
- /* last_error_time */
- if (wentry->last_error_time != 0)
- values[i++] = TimestampTzGetDatum(wentry->last_error_time);
+ /* stats_reset */
+ if (subentry->stat_reset_timestamp == 0)
+ nulls[3] = true;
else
- nulls[i++] = true;
+ values[3] = TimestampTzGetDatum(subentry->stat_reset_timestamp);
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7f1ee97f55..811252c072 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5377,14 +5377,14 @@
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
-{ oid => '8523', descr => 'statistics: information about subscription worker',
- proname => 'pg_stat_get_subscription_worker', prorows => '1', proisstrict => 'f',
- proretset => 't', provolatile => 's', proparallel => 'r',
- prorettype => 'record', proargtypes => 'oid oid',
- proallargtypes => '{oid,oid,oid,oid,oid,text,xid,int8,text,timestamptz}',
- proargmodes => '{i,i,o,o,o,o,o,o,o,o}',
- proargnames => '{subid,subrelid,subid,subrelid,last_error_relid,last_error_command,last_error_xid,last_error_count,last_error_message,last_error_time}',
- prosrc => 'pg_stat_get_subscription_worker' },
+{ oid => '8523', descr => 'statistics: information about subscription activity',
+ proname => 'pg_stat_get_subscription_activity', proisstrict => 'f',
+ provolatile => 's', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o}',
+ proargnames => '{subid,subid,apply_error_count,sync_error_count,stats_reset}',
+ prosrc => 'pg_stat_get_subscription_activity' },
{ oid => '6118', descr => 'statistics: information about subscription',
proname => 'pg_stat_get_subscription', prorows => '10', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
@@ -5773,15 +5773,10 @@
provolatile => 'v', prorettype => 'void', proargtypes => 'text',
prosrc => 'pg_stat_reset_replication_slot' },
{ oid => '8524',
- descr => 'statistics: reset collected statistics for a single subscription worker',
- proname => 'pg_stat_reset_subscription_worker', proisstrict => 'f',
- provolatile => 'v', prorettype => 'void', proargtypes => 'oid oid',
- prosrc => 'pg_stat_reset_subscription_worker_subrel' },
-{ oid => '8525',
- descr => 'statistics: reset all collected statistics for a single subscription',
- proname => 'pg_stat_reset_subscription_worker',
+ descr => 'statistics: reset collected statistics for a single subscription',
+ proname => 'pg_stat_reset_subscription_activity', proisstrict => 'f',
provolatile => 'v', prorettype => 'void', proargtypes => 'oid',
- prosrc => 'pg_stat_reset_subscription_worker_sub' },
+ prosrc => 'pg_stat_reset_subscription_activity' },
{ oid => '3163', descr => 'current trigger depth',
proname => 'pg_trigger_depth', provolatile => 's', proparallel => 'r',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index e10d20222a..2a301621e3 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -67,6 +67,7 @@ typedef enum StatMsgType
PGSTAT_MTYPE_RESETSINGLECOUNTER,
PGSTAT_MTYPE_RESETSLRUCOUNTER,
PGSTAT_MTYPE_RESETREPLSLOTCOUNTER,
+ PGSTAT_MTYPE_RESETSUBCOUNTER,
PGSTAT_MTYPE_AUTOVAC_START,
PGSTAT_MTYPE_VACUUM,
PGSTAT_MTYPE_ANALYZE,
@@ -85,7 +86,7 @@ typedef enum StatMsgType
PGSTAT_MTYPE_CONNECT,
PGSTAT_MTYPE_DISCONNECT,
PGSTAT_MTYPE_SUBSCRIPTIONPURGE,
- PGSTAT_MTYPE_SUBWORKERERROR,
+ PGSTAT_MTYPE_SUBSCRIPTIONERROR,
} StatMsgType;
/* ----------
@@ -148,8 +149,7 @@ typedef enum PgStat_Shared_Reset_Target
typedef enum PgStat_Single_Reset_Type
{
RESET_TABLE,
- RESET_FUNCTION,
- RESET_SUBWORKER
+ RESET_FUNCTION
} PgStat_Single_Reset_Type;
/* ------------------------------------------------------------
@@ -368,7 +368,6 @@ typedef struct PgStat_MsgResetsinglecounter
Oid m_databaseid;
PgStat_Single_Reset_Type m_resettype;
Oid m_objectid;
- Oid m_subobjectid;
} PgStat_MsgResetsinglecounter;
/* ----------
@@ -394,6 +393,19 @@ typedef struct PgStat_MsgResetreplslotcounter
bool clearall;
} PgStat_MsgResetreplslotcounter;
+/* ----------
+ * PgStat_MsgResetsubcounter Sent by the backend to tell the collector
+ * to reset subscription counter(s)
+ * ----------
+ */
+typedef struct PgStat_MsgResetsubcounter
+{
+ PgStat_MsgHdr m_hdr;
+ Oid m_subid; /* InvalidOid for clearing all subscription
+ * stats */
+} PgStat_MsgResetsubcounter;
+
+
/* ----------
* PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
* that a database is going to be processed
@@ -543,52 +555,28 @@ typedef struct PgStat_MsgReplSlot
/* ----------
* PgStat_MsgSubscriptionPurge Sent by the backend and autovacuum to tell the
- * collector about the dead subscriptions.
+ * collector about the dead subscription.
* ----------
*/
-#define PGSTAT_NUM_SUBSCRIPTIONPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) / sizeof(Oid))
-
typedef struct PgStat_MsgSubscriptionPurge
{
PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_subids[PGSTAT_NUM_SUBSCRIPTIONPURGE];
+ Oid m_subid;
} PgStat_MsgSubscriptionPurge;
/* ----------
- * PgStat_MsgSubWorkerError Sent by the apply worker or the table sync
- * worker to report the error occurred while
- * processing changes.
+ * PgStat_MsgSubscriptionError Sent by the apply worker or the table sync
+ * worker to report an error on the subscription.
* ----------
*/
-#define PGSTAT_SUBWORKERERROR_MSGLEN 256
-typedef struct PgStat_MsgSubWorkerError
+typedef struct PgStat_MsgSubscriptionError
{
PgStat_MsgHdr m_hdr;
- /*
- * m_subid and m_subrelid are used to determine the subscription and the
- * reporter of the error. m_subrelid is InvalidOid if reported by an apply
- * worker otherwise reported by a table sync worker.
- */
Oid m_databaseid;
Oid m_subid;
- Oid m_subrelid;
-
- /*
- * Oid of the table that the reporter was actually processing. m_relid can
- * be InvalidOid if an error occurred during worker applying a
- * non-data-modification message such as RELATION.
- */
- Oid m_relid;
-
- LogicalRepMsgType m_command;
- TransactionId m_xid;
- TimestampTz m_timestamp;
- char m_message[PGSTAT_SUBWORKERERROR_MSGLEN];
-} PgStat_MsgSubWorkerError;
+ bool m_is_apply_error;
+} PgStat_MsgSubscriptionError;
/* ----------
* PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
@@ -750,6 +738,7 @@ typedef union PgStat_Msg
PgStat_MsgResetsinglecounter msg_resetsinglecounter;
PgStat_MsgResetslrucounter msg_resetslrucounter;
PgStat_MsgResetreplslotcounter msg_resetreplslotcounter;
+ PgStat_MsgResetsubcounter msg_resetsubcounter;
PgStat_MsgAutovacStart msg_autovacuum_start;
PgStat_MsgVacuum msg_vacuum;
PgStat_MsgAnalyze msg_analyze;
@@ -767,8 +756,8 @@ typedef union PgStat_Msg
PgStat_MsgReplSlot msg_replslot;
PgStat_MsgConnect msg_connect;
PgStat_MsgDisconnect msg_disconnect;
+ PgStat_MsgSubscriptionError msg_subscriptionerror;
PgStat_MsgSubscriptionPurge msg_subscriptionpurge;
- PgStat_MsgSubWorkerError msg_subworkererror;
} PgStat_Msg;
@@ -823,16 +812,11 @@ typedef struct PgStat_StatDBEntry
TimestampTz stats_timestamp; /* time of db stats file update */
/*
- * tables, functions, and subscription workers must be last in the struct,
- * because we don't write the pointers out to the stats file.
- *
- * subworkers is the hash table of PgStat_StatSubWorkerEntry which stores
- * statistics of logical replication workers: apply worker and table sync
- * worker.
+ * tables and functions must be last in the struct, because we don't write
+ * the pointers out to the stats file.
*/
HTAB *tables;
HTAB *functions;
- HTAB *subworkers;
} PgStat_StatDBEntry;
@@ -989,38 +973,17 @@ typedef struct PgStat_StatReplSlotEntry
TimestampTz stat_reset_timestamp;
} PgStat_StatReplSlotEntry;
-/* The lookup key for subscription worker hash table */
-typedef struct PgStat_StatSubWorkerKey
-{
- Oid subid;
-
- /*
- * Oid of the table for which tablesync worker will copy the initial data.
- * An InvalidOid will be assigned for apply workers.
- */
- Oid subrelid;
-} PgStat_StatSubWorkerKey;
-
/*
- * Logical replication apply worker and table sync worker statistics kept in the
- * stats collector.
+ * Subscription statistics kept in the stats collector.
*/
-typedef struct PgStat_StatSubWorkerEntry
+typedef struct PgStat_StatSubEntry
{
- PgStat_StatSubWorkerKey key; /* hash key (must be first) */
+ Oid subid; /* hash key (must be first) */
- /*
- * Subscription worker error statistics representing an error that
- * occurred during application of changes or the initial table
- * synchronization.
- */
- Oid last_error_relid;
- LogicalRepMsgType last_error_command;
- TransactionId last_error_xid;
- PgStat_Counter last_error_count;
- TimestampTz last_error_time;
- char last_error_message[PGSTAT_SUBWORKERERROR_MSGLEN];
-} PgStat_StatSubWorkerEntry;
+ PgStat_Counter apply_error_count;
+ PgStat_Counter sync_error_count;
+ TimestampTz stat_reset_timestamp;
+} PgStat_StatSubEntry;
/*
* Working state needed to accumulate per-function-call timing statistics.
@@ -1111,10 +1074,10 @@ extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
extern void pgstat_reset_shared_counters(const char *);
-extern void pgstat_reset_single_counter(Oid objectid, Oid subobjectid,
- PgStat_Single_Reset_Type type);
+extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_reset_slru_counter(const char *);
extern void pgstat_reset_replslot_counter(const char *name);
+extern void pgstat_reset_subscription_counter(Oid subid);
extern void pgstat_report_connect(Oid dboid);
extern void pgstat_report_autovac(Oid dboid);
@@ -1131,9 +1094,7 @@ extern void pgstat_report_checksum_failure(void);
extern void pgstat_report_replslot(const PgStat_StatReplSlotEntry *repSlotStat);
extern void pgstat_report_replslot_create(const char *slotname);
extern void pgstat_report_replslot_drop(const char *slotname);
-extern void pgstat_report_subworker_error(Oid subid, Oid subrelid, Oid relid,
- LogicalRepMsgType command,
- TransactionId xid, const char *errmsg);
+extern void pgstat_report_subscription_error(Oid subid, bool is_apply_error);
extern void pgstat_report_subscription_drop(Oid subid);
extern void pgstat_initialize(void);
@@ -1226,8 +1187,7 @@ extern void pgstat_send_wal(bool force);
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
-extern PgStat_StatSubWorkerEntry *pgstat_fetch_stat_subworker_entry(Oid subid,
- Oid subrelid);
+extern PgStat_StatSubEntry *pgstat_fetch_stat_subscription(Oid subid);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_BgWriterStats *pgstat_fetch_stat_bgwriter(void);
extern PgStat_CheckpointerStats *pgstat_fetch_stat_checkpointer(void);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 1420288d67..1fe4a98d0e 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2072,24 +2072,13 @@ pg_stat_subscription| SELECT su.oid AS subid,
st.latest_end_time
FROM (pg_subscription su
LEFT JOIN pg_stat_get_subscription(NULL::oid) st(subid, relid, pid, received_lsn, last_msg_send_time, last_msg_receipt_time, latest_end_lsn, latest_end_time) ON ((st.subid = su.oid)));
-pg_stat_subscription_workers| SELECT w.subid,
+pg_stat_subscription_activity| SELECT a.subid,
s.subname,
- w.subrelid,
- w.last_error_relid,
- w.last_error_command,
- w.last_error_xid,
- w.last_error_count,
- w.last_error_message,
- w.last_error_time
- FROM ( SELECT pg_subscription.oid AS subid,
- NULL::oid AS relid
- FROM pg_subscription
- UNION ALL
- SELECT pg_subscription_rel.srsubid AS subid,
- pg_subscription_rel.srrelid AS relid
- FROM pg_subscription_rel) sr,
- (LATERAL pg_stat_get_subscription_worker(sr.subid, sr.relid) w(subid, subrelid, last_error_relid, last_error_command, last_error_xid, last_error_count, last_error_message, last_error_time)
- JOIN pg_subscription s ON ((w.subid = s.oid)));
+ a.apply_error_count,
+ a.sync_error_count,
+ a.stats_reset
+ FROM pg_subscription s,
+ LATERAL pg_stat_get_subscription_activity(s.oid) a(subid, apply_error_count, sync_error_count, stats_reset);
pg_stat_sys_indexes| SELECT pg_stat_all_indexes.relid,
pg_stat_all_indexes.indexrelid,
pg_stat_all_indexes.schemaname,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
new file mode 100644
index 0000000000..72d0943383
--- /dev/null
+++ b/src/test/subscription/t/026_stats.pl
@@ -0,0 +1,102 @@
+
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+# Tests for subscription stats.
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Create publisher node.
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+# Create subscriber node.
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+# Initial table setup on both publisher and subscriber. On subscriber we
+# create the same tables but with primary keys. Also, insert some data that
+# will conflict with the data replicated from publisher later.
+$node_publisher->safe_psql(
+ 'postgres',
+ qq[
+BEGIN;
+CREATE TABLE test_tab1 (a int);
+INSERT INTO test_tab1 VALUES (1);
+COMMIT;
+]);
+$node_subscriber->safe_psql(
+ 'postgres',
+ qq[
+BEGIN;
+CREATE TABLE test_tab1 (a int primary key);
+INSERT INTO test_tab1 VALUES (1);
+COMMIT;
+]);
+
+# Setup publication.
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION tap_pub FOR TABLE test_tab1;");
+
+# There shouldn't be any subscription errors before starting logical replication.
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(1) FROM pg_stat_subscription_activity");
+is($result, qq(0), 'check no subscription error');
+
+# Create subscription. The tablesync for test_tab1 on tap_sub will enter into
+# infinite error loop due to violating the unique constraint.
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub;"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Wait for the tablesync error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT sync_error_count > 0
+FROM pg_stat_subscription_activity
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for tablesync error";
+
+# Truncate test_tab1 so that tablesync worker can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
+
+# Wait for initial tablesync for test_tab1 to finish.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT count(1) = 1 FROM pg_subscription_rel
+WHERE srrelid = 'test_tab1'::regclass AND srsubstate in ('r', 's')
+]) or die "Timed out while waiting for subscriber to synchronize data";
+
+# Check test_tab1 on the subscriber has one row.
+$result = $node_subscriber->safe_psql('postgres', "SELECT a FROM test_tab1");
+is($result, qq(1), 'check the table has now row');
+
+# Insert data to test_tab1 on the publisher, raising an error on the subscriber
+# due to violation of the unique constraint on test_tab1.
+$node_publisher->safe_psql('postgres', "INSERT INTO test_tab1 VALUES (1)");
+
+# Wait for the apply error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT apply_error_count > 0
+FROM pg_stat_subscription_activity
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for apply error";
+
+# Truncate test_tab1 so that apply worker can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
+
+$node_subscriber->stop('fast');
+$node_publisher->stop('fast');
+
+done_testing();
diff --git a/src/test/subscription/t/026_worker_stats.pl b/src/test/subscription/t/026_worker_stats.pl
deleted file mode 100644
index f72e4766e8..0000000000
--- a/src/test/subscription/t/026_worker_stats.pl
+++ /dev/null
@@ -1,165 +0,0 @@
-
-# Copyright (c) 2021-2022, PostgreSQL Global Development Group
-
-# Tests for subscription error stats.
-use strict;
-use warnings;
-use PostgreSQL::Test::Cluster;
-use PostgreSQL::Test::Utils;
-use Test::More;
-
-# Test if the error reported on pg_stat_subscription_workers view is expected.
-sub test_subscription_error
-{
- my ($node, $relname, $command, $xid, $by_apply_worker, $errmsg_prefix, $msg)
- = @_;
-
- my $check_sql = qq[
-SELECT count(1) > 0
-FROM pg_stat_subscription_workers
-WHERE last_error_relid = '$relname'::regclass
- AND starts_with(last_error_message, '$errmsg_prefix')];
-
- # subrelid
- $check_sql .= $by_apply_worker
- ? qq[ AND subrelid IS NULL]
- : qq[ AND subrelid = '$relname'::regclass];
-
- # last_error_command
- $check_sql .= $command eq ''
- ? qq[ AND last_error_command IS NULL]
- : qq[ AND last_error_command = '$command'];
-
- # last_error_xid
- $check_sql .= $xid eq ''
- ? qq[ AND last_error_xid IS NULL]
- : qq[ AND last_error_xid = '$xid'::xid];
-
- # Wait for the particular error statistics to be reported.
- $node->poll_query_until('postgres', $check_sql,
-) or die "Timed out while waiting for " . $msg;
-}
-
-# Create publisher node.
-my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
-$node_publisher->init(allows_streaming => 'logical');
-$node_publisher->start;
-
-# Create subscriber node.
-my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
-$node_subscriber->init(allows_streaming => 'logical');
-
-# The subscriber will enter an infinite error loop, so we don't want
-# to overflow the server log with error messages.
-$node_subscriber->append_conf('postgresql.conf',
- qq[
-wal_retrieve_retry_interval = 2s
-]);
-$node_subscriber->start;
-
-# Initial table setup on both publisher and subscriber. On subscriber we
-# create the same tables but with primary keys. Also, insert some data that
-# will conflict with the data replicated from publisher later.
-$node_publisher->safe_psql(
- 'postgres',
- qq[
-BEGIN;
-CREATE TABLE test_tab1 (a int);
-CREATE TABLE test_tab2 (a int);
-INSERT INTO test_tab1 VALUES (1);
-INSERT INTO test_tab2 VALUES (1);
-COMMIT;
-]);
-$node_subscriber->safe_psql(
- 'postgres',
- qq[
-BEGIN;
-CREATE TABLE test_tab1 (a int primary key);
-CREATE TABLE test_tab2 (a int primary key);
-INSERT INTO test_tab2 VALUES (1);
-COMMIT;
-]);
-
-# Setup publications.
-my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
-$node_publisher->safe_psql(
- 'postgres',
- "CREATE PUBLICATION tap_pub FOR TABLE test_tab1, test_tab2;");
-
-# There shouldn't be any subscription errors before starting logical replication.
-my $result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(1) FROM pg_stat_subscription_workers");
-is($result, qq(0), 'check no subscription error');
-
-# Create subscription. The table sync for test_tab2 on tap_sub will enter into
-# infinite error loop due to violating the unique constraint.
-$node_subscriber->safe_psql(
- 'postgres',
- "CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub;");
-
-$node_publisher->wait_for_catchup('tap_sub');
-
-# Wait for initial table sync for test_tab1 to finish.
-$node_subscriber->poll_query_until(
- 'postgres',
- qq[
-SELECT count(1) = 1 FROM pg_subscription_rel
-WHERE srrelid = 'test_tab1'::regclass AND srsubstate in ('r', 's')
-]) or die "Timed out while waiting for subscriber to synchronize data";
-
-# Check the initial data.
-$result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(a) FROM test_tab1");
-is($result, q(1), 'check initial data are copied to subscriber');
-
-# Insert more data to test_tab1, raising an error on the subscriber due to
-# violation of the unique constraint on test_tab1.
-my $xid = $node_publisher->safe_psql(
- 'postgres',
- qq[
-BEGIN;
-INSERT INTO test_tab1 VALUES (1);
-SELECT pg_current_xact_id()::xid;
-COMMIT;
-]);
-test_subscription_error($node_subscriber, 'test_tab1', 'INSERT', $xid,
- 1, # check apply worker error
- qq(duplicate key value violates unique constraint),
- 'error reported by the apply worker');
-
-# Check the table sync worker's error in the view.
-test_subscription_error($node_subscriber, 'test_tab2', '', '',
- 0, # check tablesync worker error
- qq(duplicate key value violates unique constraint),
- 'the error reported by the table sync worker');
-
-# Test for resetting subscription worker statistics.
-# Truncate test_tab1 and test_tab2 so that applying changes and table sync can
-# continue, respectively.
-$node_subscriber->safe_psql(
- 'postgres',
- "TRUNCATE test_tab1, test_tab2;");
-
-# Wait for the data to be replicated.
-$node_subscriber->poll_query_until(
- 'postgres',
- "SELECT count(1) > 0 FROM test_tab1");
-$node_subscriber->poll_query_until(
- 'postgres',
- "SELECT count(1) > 0 FROM test_tab2");
-
-# There shouldn't be any errors in the view after dropping the subscription.
-$node_subscriber->safe_psql(
- 'postgres',
- "DROP SUBSCRIPTION tap_sub;");
-$result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(1) FROM pg_stat_subscription_workers");
-is($result, q(0), 'no error after dropping subscription');
-
-$node_subscriber->stop('fast');
-$node_publisher->stop('fast');
-
-done_testing();
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 15684f53ba..94b099df5e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1944,8 +1944,8 @@ PgStat_MsgResetsharedcounter
PgStat_MsgResetsinglecounter
PgStat_MsgResetslrucounter
PgStat_MsgSLRU
+PgStat_MsgSubscriptionError
PgStat_MsgSubscriptionPurge
-PgStat_MsgSubWorkerError
PgStat_MsgTabpurge
PgStat_MsgTabstat
PgStat_MsgTempFile
@@ -1957,8 +1957,7 @@ PgStat_Single_Reset_Type
PgStat_StatDBEntry
PgStat_StatFuncEntry
PgStat_StatReplSlotEntry
-PgStat_StatSubWorkerEntry
-PgStat_StatSubWorkerKey
+PgStat_StatSubEntry
PgStat_StatTabEntry
PgStat_SubXactStatus
PgStat_TableCounts
--
2.24.3 (Apple Git-128)
^ permalink raw reply [nested|flat] 42+ messages in thread
* RE: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 07:20 [email protected] <[email protected]>
parent: Masahiko Sawada <[email protected]>
1 sibling, 1 reply; 42+ messages in thread
From: [email protected] @ 2022-02-24 07:20 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; Peter Smith <[email protected]>; +Cc: Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Thu, Feb 24, 2022 9:33 AM Masahiko Sawada <[email protected]> wrote:
>
> Thank you for the comments! I've attached the latest version patch
> that incorporated all comments I got so far. The primary change from
> the previous version is that the subscription statistics live globally
> rather than per-database.
>
Thanks for updating the patch.
Few comments:
1.
I think we should add some doc for column stats_reset in pg_stat_subscription_activity view.
2.
+CREATE VIEW pg_stat_subscription_activity AS
SELECT
- w.subid,
+ a.subid,
s.subname,
...
+ a.apply_error_count,
+ a.sync_error_count,
+ a.stats_reset
+ FROM pg_subscription as s,
+ pg_stat_get_subscription_activity(oid) as a;
The line "a.stats_reset" uses a Tab, and we'd better use spaces here.
Regards,
Tang
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 08:53 Peter Smith <[email protected]>
parent: [email protected] <[email protected]>
0 siblings, 2 replies; 42+ messages in thread
From: Peter Smith @ 2022-02-24 08:53 UTC (permalink / raw)
To: [email protected] <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Hi. Below are my review comments for the v2 patch.
======
1. Commit message
This patch changes the pg_stat_subscription_workers view (introduced
by commit 8d74fc9) so that it stores only statistics counters:
apply_error_count and sync_error_count, and has one entry for
subscription.
SUGGESTION
"and has one entry for subscription." --> "and has one entry for each
subscription."
~~~
2. Commit message
After removing these error details, there are no longer relation
information, so the subscription statistics are now a cluster-wide
statistics.
SUGGESTION
"there are no longer relation information," --> "there is no longer
any relation information,"
~~~
3. doc/src/sgml/monitoring.sgml
- <para>
- The error message
+ Number of times the error occurred during the application of changes
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_time</structfield> <type>timestamp
with time zone</type>
+ <structfield>sync_error_count</structfield> <type>bigint</type>
</para>
<para>
- Last time at which this error occurred
+ Number of times the error occurred during the initial table
+ synchronization
</para></entry>
SUGGESTION (both places)
"Number of times the error occurred ..." --> "Number of times an error
occurred ..."
~~~
4. doc/src/sgml/monitoring.sgml - missing column
(duplicate - also reported by [Tang-v2])
The PG docs for the new "stats_reset" column are missing.
~~~
5. src/backend/catalog/system_views.sql - whitespace
(duplicate - also reported by [Tang-v2])
- JOIN pg_subscription s ON (w.subid = s.oid);
+ a.apply_error_count,
+ a.sync_error_count,
+ a.stats_reset
+ FROM pg_subscription as s,
+ pg_stat_get_subscription_activity(oid) as a;
inconsistent tab/space indenting for 'a.stats_reset'.
~~~
6. src/backend/postmaster/pgstat.c - function name
+/* ----------
+ * pgstat_reset_subscription_counter() -
+ *
+ * Tell the statistics collector to reset a single subscription
+ * counter, or all subscription counters (when subid is InvalidOid).
+ *
+ * Permission checking for this function is managed through the normal
+ * GRANT system.
+ * ----------
+ */
+void
+pgstat_reset_subscription_counter(Oid subid)
SUGGESTION (function name)
"pgstat_reset_subscription_counter" -->
"pgstat_reset_subscription_counters" (plural)
~~
7. src/backend/postmaster/pgstat.c - pgstat_recv_resetsubcounter
@@ -5645,6 +5598,51 @@
pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg,
}
}
+/* ----------
+ * pgstat_recv_resetsubcounter() -
+ *
+ * Reset some subscription statistics of the cluster.
+ * ----------
+ */
+static void
+pgstat_recv_resetsubcounter(PgStat_MsgResetsubcounter *msg, int len)
"Reset some" seems a bit vague. Why not describe that it is all or
none according to the msg->m_subid?
~~~
8. src/backend/postmaster/pgstat.c - pgstat_recv_resetsubcounter
+ if (!OidIsValid(msg->m_subid))
+ {
+ HASH_SEQ_STATUS sstat;
+
+ /* Clear all subscription counters */
+ hash_seq_init(&sstat, subscriptionStatHash);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&sstat)) != NULL)
+ pgstat_reset_subscription(subentry, ts);
+ }
+ else
+ {
+ /* Get the subscription statistics to reset */
+ subentry = pgstat_get_subscription_entry(msg->m_subid, false);
+
+ /*
+ * Nothing to do if the given subscription entry is not found. This
+ * could happen when the subscription with the subid is removed and
+ * the corresponding statistics entry is also removed before receiving
+ * the reset message.
+ */
+ if (!subentry)
+ return;
+
+ /* Reset the stats for the requested replication slot */
+ pgstat_reset_subscription(subentry, ts);
+ }
+}
Why not reverse the if/else?
Checking OidIsValid(...) seems more natural than checking !OidIsValid(...)
~~~
9. src/backend/postmaster/pgstat.c - pgstat_recv_subscription_purge
static void
pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len)
{
/* Return if we don't have replication subscription statistics */
if (subscriptionStatHash == NULL)
return;
/* Remove from hashtable if present; we don't care if it's not */
(void) hash_search(subscriptionStatHash, (void *) &(msg->m_subid),
HASH_REMOVE, NULL);
}
SUGGESTION
Wouldn't the above code be simpler written like:
if (subscriptionStatHash)
{
/* Remove from hashtable if present; we don't care if it's not */
(void) hash_search(subscriptionStatHash, (void *) &(msg->m_subid),
HASH_REMOVE, NULL);
}
~~~
10. src/backend/replication/logical/worker.c
(from my previous [Peter-v1] #9)
>> + /* Report the worker failed during table synchronization */
>> + pgstat_report_subscription_error(MyLogicalRepWorker->subid, false);
>>
>> and
>>
>> + /* Report the worker failed during the application of the change */
>> + pgstat_report_subscription_error(MyLogicalRepWorker->subid, true);
>>
>>
>> Why don't these use MySubscription->oid instead of MyLogicalRepWorker->subid?
> It's just because we used to use MyLogicalRepWorker->subid, is there
> any particular reason why we should use MySubscription->oid here?
I felt MySubscription->oid is a more natural and more direct way of
expressing the same thing.
Consider: "the oid of the current subscription" versus "the oid of
the subscription of the current worker". IMO the first one is simpler.
YMMV.
Also, it is shorter :)
~~~
11. src/include/pgstat.h - enum order
(follow-on from my previous v1 review comment #10)
>I assume you're concerned about binary compatibility or something. I
>think it should not be a problem since both
>PGSTAT_MTYPE_SUBWORKERERROR and PGSTAT_MTYPE_SUBSCRIPTIONPURGE are
>introduced to PG15.
Yes, maybe it is OK for those ones. But now in v2 there is a new
PGSTAT_MTYPE_RESETSUBCOUNTER.
Shouldn't at least that one be put at the end for the same reason?
~~~
12. src/include/pgstat.h - PgStat_MsgResetsubcounter
Maybe a better name for this is "PgStat_MsgResetsubcounters" (plural)?
~~~
13. src/test/subscription/t/026_worker_stats.pl - missing test?
Shouldn't there also be some test to reset the counters to confirm
that they really do get reset to zero?
~~~
14. src/tools/pgindent/typedefs.list
PgStat_MsgResetsubcounter (from pgstat.h) is missing?
------
15. pg_stat_subscription_activity view name?
Has the view name already been decided or still under discussion - I
was not sure?
If is it already decided then fine, but if not then my vote would be
for something different like:
e.g.1 - pg_stat_subscription_errors
e.g.2 - pg_stat_subscription_counters
e.g.3 - pg_stat_subscription_metrics
Maybe "activity" was chosen to be deliberately vague in case some
future unknown stats columns get added? But it means now there is a
corresponding function "pg_stat_reset_subscription_activity", when in
practice you don't really reset activity - what you want to do is
reset some statistics *about* the activity... so it all seems a bit
odd to me.
------
[Tang-v2] https://www.postgresql.org/message-id/OS0PR01MB6113769B17E90ADC9ACA14B2FB3D9%40OS0PR01MB6113.jpnprd0...
[Peter-v1] https://www.postgresql.org/message-id/CAHut%2BPtH-uN5rbGRh-%3DkCd8xvQYDf_JCcjLcVjW3OXGz6T%2BxCw%40ma...
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 09:20 Peter Eisentraut <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Peter Eisentraut @ 2022-02-24 09:20 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Amit Kapila <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On 21.02.22 17:17, Andres Freund wrote:
> Hi,
>
> On 2022-02-21 14:49:01 +0530, Amit Kapila wrote:
>> On Mon, Feb 21, 2022 at 1:18 PM Andres Freund <[email protected]> wrote:
>>>> * stats_reset (Time at which these statistics were last reset)
>>>>
>>>> The view name could be pg_stat_subscription_lrep,
>>>> pg_stat_logical_replication, or something on those lines.
>>>
>>> pg_stat_subscription_stats :)
>
>> Having *stat* two times in the name sounds slightly odd to me but let
>> us see what others think. One more option could be
>> pg_stat_subscription_replication.
>
> It was a joke, making light of our bad naming in pg_stat_*, not a serious
> suggestion...
I think pg_stat_subscription_stats is actually the least worst option.
Unless we want to consider renaming pg_stat_subscription (which is
actually more like pg_stat_subscription_activity). But I think that
should be avoided.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 09:48 Peter Eisentraut <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 0 replies; 42+ messages in thread
From: Peter Eisentraut @ 2022-02-24 09:48 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Masahiko Sawada <[email protected]>; +Cc: Amit Kapila <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On 23.02.22 03:14, Andres Freund wrote:
> Why are the stats stored in the per-database stats file / as a second level
> below the database? While they're also associated with a database, it's a
> global catalog, so it seems to make more sense to have them "live" globally as
> well?
pg_subscription being a global catalog is a bit of a lie for the benefit
of the worker launcher, but it can be treated as a per-database catalog
for practical purposes.
> Not just from an aesthetical perspective, but there might also be cases where
> it's useful to send stats from the stats launcher. E.g. the number of times
> the launcher couldn't start a worker because the max numbers of workers was
> already active or such.
That's a reasonable point, however.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 09:53 Peter Eisentraut <[email protected]>
parent: Masahiko Sawada <[email protected]>
1 sibling, 1 reply; 42+ messages in thread
From: Peter Eisentraut @ 2022-02-24 09:53 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; Peter Smith <[email protected]>; +Cc: Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On 24.02.22 02:32, Masahiko Sawada wrote:
> On Wed, Feb 23, 2022 at 12:08 PM Peter Smith <[email protected]> wrote:
>>
>> Hi. Below are my review comments for the v1 patch.
>
> Thank you for the comments! I've attached the latest version patch
> that incorporated all comments I got so far. The primary change from
> the previous version is that the subscription statistics live globally
> rather than per-database.
I don't think the name pg_stat_subscription_activity is a good choice.
We have a view called pg_stat_activity, which is very well known. From
that perspective, "activity" means what is happening right now or what
has happened most recently. The reworked view in this patch does not
contain that (we already have pg_stat_subscription for that), but it
contains accumulated counters.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 11:46 Masahiko Sawada <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 1 reply; 42+ messages in thread
From: Masahiko Sawada @ 2022-02-24 11:46 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Thu, Feb 24, 2022 at 6:53 PM Peter Eisentraut
<[email protected]> wrote:
>
> On 24.02.22 02:32, Masahiko Sawada wrote:
> > On Wed, Feb 23, 2022 at 12:08 PM Peter Smith <[email protected]> wrote:
> >>
> >> Hi. Below are my review comments for the v1 patch.
> >
> > Thank you for the comments! I've attached the latest version patch
> > that incorporated all comments I got so far. The primary change from
> > the previous version is that the subscription statistics live globally
> > rather than per-database.
>
> I don't think the name pg_stat_subscription_activity is a good choice.
>
> We have a view called pg_stat_activity, which is very well known. From
> that perspective, "activity" means what is happening right now or what
> has happened most recently. The reworked view in this patch does not
> contain that (we already have pg_stat_subscription for that), but it
> contains accumulated counters.
Right.
What pg_stat_subscription shows is rather suitable for the name
pg_stat_subscription_activity than the reworked view. But switching
these names would also not be a good idea. I think it's better to use
"subscription" in the view name since it shows actually statistics for
subscriptions and subscription OID is the key. I personally prefer
pg_stat_subscription_counters among the ideas that have been proposed
so far, but I'd like to hear opinions and votes.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 12:05 Amit Kapila <[email protected]>
parent: Peter Smith <[email protected]>
1 sibling, 1 reply; 42+ messages in thread
From: Amit Kapila @ 2022-02-24 12:05 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: [email protected] <[email protected]>; Masahiko Sawada <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Thu, Feb 24, 2022 at 2:24 PM Peter Smith <[email protected]> wrote:
>
> 9. src/backend/postmaster/pgstat.c - pgstat_recv_subscription_purge
>
> static void
> pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len)
> {
> /* Return if we don't have replication subscription statistics */
> if (subscriptionStatHash == NULL)
> return;
>
> /* Remove from hashtable if present; we don't care if it's not */
> (void) hash_search(subscriptionStatHash, (void *) &(msg->m_subid),
> HASH_REMOVE, NULL);
> }
>
> SUGGESTION
> Wouldn't the above code be simpler written like:
>
> if (subscriptionStatHash)
> {
> /* Remove from hashtable if present; we don't care if it's not */
> (void) hash_search(subscriptionStatHash, (void *) &(msg->m_subid),
> HASH_REMOVE, NULL);
> }
> ~~~
>
I think we can write that way as well but I would prefer the way it is
currently in the patch as we use a similar pattern in nearby code (ex.
pgstat_recv_resetreplslotcounter) and at other places in the code base
as well.
> 10. src/backend/replication/logical/worker.c
>
> (from my previous [Peter-v1] #9)
>
> >> + /* Report the worker failed during table synchronization */
> >> + pgstat_report_subscription_error(MyLogicalRepWorker->subid, false);
> >>
> >> and
> >>
> >> + /* Report the worker failed during the application of the change */
> >> + pgstat_report_subscription_error(MyLogicalRepWorker->subid, true);
> >>
> >>
> >> Why don't these use MySubscription->oid instead of MyLogicalRepWorker->subid?
>
> > It's just because we used to use MyLogicalRepWorker->subid, is there
> > any particular reason why we should use MySubscription->oid here?
>
> I felt MySubscription->oid is a more natural and more direct way of
> expressing the same thing.
>
> Consider: "the oid of the current subscription" versus "the oid of
> the subscription of the current worker". IMO the first one is simpler.
> YMMV.
>
I think we can use either but maybe MySubscription->oid would be
slightly better here as the same is used in nearby code as well.
> Also, it is shorter :)
>
> ~~~
>
> 11. src/include/pgstat.h - enum order
>
> (follow-on from my previous v1 review comment #10)
>
> >I assume you're concerned about binary compatibility or something. I
> >think it should not be a problem since both
> >PGSTAT_MTYPE_SUBWORKERERROR and PGSTAT_MTYPE_SUBSCRIPTIONPURGE are
> >introduced to PG15.
>
> Yes, maybe it is OK for those ones. But now in v2 there is a new
> PGSTAT_MTYPE_RESETSUBCOUNTER.
>
> Shouldn't at least that one be put at the end for the same reason?
>
> ~~~
>
I don't see the reason to put that at end. It is better to add it near
to similar RESET enums.
>
> 13. src/test/subscription/t/026_worker_stats.pl - missing test?
>
> Shouldn't there also be some test to reset the counters to confirm
> that they really do get reset to zero?
>
> ~~~
>
I think we avoid writing tests for stats for each and every case as it
is not reliable in nature (the message can be lost). If we can find a
reliable way then it is okay.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 12:17 Masahiko Sawada <[email protected]>
parent: Amit Kapila <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Masahiko Sawada @ 2022-02-24 12:17 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Peter Smith <[email protected]>; [email protected] <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Thu, Feb 24, 2022 at 9:05 PM Amit Kapila <[email protected]> wrote:
>
> On Thu, Feb 24, 2022 at 2:24 PM Peter Smith <[email protected]> wrote:
> >
> > 10. src/backend/replication/logical/worker.c
> >
> > (from my previous [Peter-v1] #9)
> >
> > >> + /* Report the worker failed during table synchronization */
> > >> + pgstat_report_subscription_error(MyLogicalRepWorker->subid, false);
> > >>
> > >> and
> > >>
> > >> + /* Report the worker failed during the application of the change */
> > >> + pgstat_report_subscription_error(MyLogicalRepWorker->subid, true);
> > >>
> > >>
> > >> Why don't these use MySubscription->oid instead of MyLogicalRepWorker->subid?
> >
> > > It's just because we used to use MyLogicalRepWorker->subid, is there
> > > any particular reason why we should use MySubscription->oid here?
> >
> > I felt MySubscription->oid is a more natural and more direct way of
> > expressing the same thing.
> >
> > Consider: "the oid of the current subscription" versus "the oid of
> > the subscription of the current worker". IMO the first one is simpler.
> > YMMV.
> >
>
> I think we can use either but maybe MySubscription->oid would be
> slightly better here as the same is used in nearby code as well.
Okay, will change.
> >
> > 13. src/test/subscription/t/026_worker_stats.pl - missing test?
> >
> > Shouldn't there also be some test to reset the counters to confirm
> > that they really do get reset to zero?
> >
> > ~~~
> >
>
> I think we avoid writing tests for stats for each and every case as it
> is not reliable in nature (the message can be lost). If we can find a
> reliable way then it is okay.
Yeah, the messages can even be out-of-order. Particularly, in this
test, the apply worker and table sync worker keep reporting the
messages, it's quite possible that the test becomes unstable. I
remember we removed unstable tests of resetting statistics before
(e.g., see fc6950913).
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 12:23 Peter Eisentraut <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 2 replies; 42+ messages in thread
From: Peter Eisentraut @ 2022-02-24 12:23 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On 24.02.22 12:46, Masahiko Sawada wrote:
>> We have a view called pg_stat_activity, which is very well known. From
>> that perspective, "activity" means what is happening right now or what
>> has happened most recently. The reworked view in this patch does not
>> contain that (we already have pg_stat_subscription for that), but it
>> contains accumulated counters.
> Right.
>
> What pg_stat_subscription shows is rather suitable for the name
> pg_stat_subscription_activity than the reworked view. But switching
> these names would also not be a good idea. I think it's better to use
> "subscription" in the view name since it shows actually statistics for
> subscriptions and subscription OID is the key. I personally prefer
> pg_stat_subscription_counters among the ideas that have been proposed
> so far, but I'd like to hear opinions and votes.
_counters will fail if there is something not a counter (such as
last-timestamp-of-something).
Earlier, pg_stat_subscription_stats was mentioned, which doesn't have
that problem.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 12:51 Masahiko Sawada <[email protected]>
parent: Peter Eisentraut <[email protected]>
1 sibling, 0 replies; 42+ messages in thread
From: Masahiko Sawada @ 2022-02-24 12:51 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Peter Smith <[email protected]>; Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Thu, Feb 24, 2022 at 9:23 PM Peter Eisentraut
<[email protected]> wrote:
>
>
> On 24.02.22 12:46, Masahiko Sawada wrote:
> >> We have a view called pg_stat_activity, which is very well known. From
> >> that perspective, "activity" means what is happening right now or what
> >> has happened most recently. The reworked view in this patch does not
> >> contain that (we already have pg_stat_subscription for that), but it
> >> contains accumulated counters.
> > Right.
> >
> > What pg_stat_subscription shows is rather suitable for the name
> > pg_stat_subscription_activity than the reworked view. But switching
> > these names would also not be a good idea. I think it's better to use
> > "subscription" in the view name since it shows actually statistics for
> > subscriptions and subscription OID is the key. I personally prefer
> > pg_stat_subscription_counters among the ideas that have been proposed
> > so far, but I'd like to hear opinions and votes.
>
> _counters will fail if there is something not a counter (such as
> last-timestamp-of-something).
>
> Earlier, pg_stat_subscription_stats was mentioned, which doesn't have
> that problem.
Ah, I had misunderstood your comment. Right, _counter could be a
blocker for the future changes.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 14:45 Andres Freund <[email protected]>
parent: Peter Eisentraut <[email protected]>
1 sibling, 0 replies; 42+ messages in thread
From: Andres Freund @ 2022-02-24 14:45 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Peter Smith <[email protected]>; Amit Kapila <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Hi,
On 2022-02-24 13:23:55 +0100, Peter Eisentraut wrote:
> On 24.02.22 12:46, Masahiko Sawada wrote:
> > > We have a view called pg_stat_activity, which is very well known. From
> > > that perspective, "activity" means what is happening right now or what
> > > has happened most recently. The reworked view in this patch does not
> > > contain that (we already have pg_stat_subscription for that), but it
> > > contains accumulated counters.
> > Right.
> >
> > What pg_stat_subscription shows is rather suitable for the name
> > pg_stat_subscription_activity than the reworked view. But switching
> > these names would also not be a good idea. I think it's better to use
> > "subscription" in the view name since it shows actually statistics for
> > subscriptions and subscription OID is the key. I personally prefer
> > pg_stat_subscription_counters among the ideas that have been proposed
> > so far, but I'd like to hear opinions and votes.
>
> _counters will fail if there is something not a counter (such as
> last-timestamp-of-something).
>
> Earlier, pg_stat_subscription_stats was mentioned, which doesn't have that
> problem.
We really should try to fix this in a more general way at some point. We have
way too many different things mixed up in pg_stat_*.
I'd like to get something like the patch in soon though, we can still change
the name later. I've been blocked behind this stuff for weeks, and it's
getting really painful...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-24 15:49 Masahiko Sawada <[email protected]>
parent: Peter Smith <[email protected]>
1 sibling, 3 replies; 42+ messages in thread
From: Masahiko Sawada @ 2022-02-24 15:49 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: [email protected] <[email protected]>; Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Hi,
Thank you for the comments!
On Thu, Feb 24, 2022 at 4:20 PM [email protected]
<[email protected]> wrote:
>
> On Thu, Feb 24, 2022 9:33 AM Masahiko Sawada <[email protected]> wrote:
> >
> > Thank you for the comments! I've attached the latest version patch
> > that incorporated all comments I got so far. The primary change from
> > the previous version is that the subscription statistics live globally
> > rather than per-database.
> >
>
> Thanks for updating the patch.
>
> Few comments:
>
> 1.
> I think we should add some doc for column stats_reset in pg_stat_subscription_activity view.
Added.
>
> 2.
> +CREATE VIEW pg_stat_subscription_activity AS
> SELECT
> - w.subid,
> + a.subid,
> s.subname,
> ...
> + a.apply_error_count,
> + a.sync_error_count,
> + a.stats_reset
> + FROM pg_subscription as s,
> + pg_stat_get_subscription_activity(oid) as a;
>
> The line "a.stats_reset" uses a Tab, and we'd better use spaces here.
Fixed.
On Thu, Feb 24, 2022 at 5:54 PM Peter Smith <[email protected]> wrote:
>
> Hi. Below are my review comments for the v2 patch.
>
> ======
>
> 1. Commit message
>
> This patch changes the pg_stat_subscription_workers view (introduced
> by commit 8d74fc9) so that it stores only statistics counters:
> apply_error_count and sync_error_count, and has one entry for
> subscription.
>
> SUGGESTION
> "and has one entry for subscription." --> "and has one entry for each
> subscription."
Fixed.
>
> ~~~
>
> 2. Commit message
>
> After removing these error details, there are no longer relation
> information, so the subscription statistics are now a cluster-wide
> statistics.
>
> SUGGESTION
> "there are no longer relation information," --> "there is no longer
> any relation information,"
Fixed.
>
> ~~~
>
> 3. doc/src/sgml/monitoring.sgml
>
> - <para>
> - The error message
> + Number of times the error occurred during the application of changes
> </para></entry>
> </row>
>
> <row>
> <entry role="catalog_table_entry"><para role="column_definition">
> - <structfield>last_error_time</structfield> <type>timestamp
> with time zone</type>
> + <structfield>sync_error_count</structfield> <type>bigint</type>
> </para>
> <para>
> - Last time at which this error occurred
> + Number of times the error occurred during the initial table
> + synchronization
> </para></entry>
>
> SUGGESTION (both places)
> "Number of times the error occurred ..." --> "Number of times an error
> occurred ..."
Fixed.
>
> ~~~
>
> 4. doc/src/sgml/monitoring.sgml - missing column
>
> (duplicate - also reported by [Tang-v2])
>
> The PG docs for the new "stats_reset" column are missing.
>
> ~~~
>
> 5. src/backend/catalog/system_views.sql - whitespace
>
> (duplicate - also reported by [Tang-v2])
>
> - JOIN pg_subscription s ON (w.subid = s.oid);
> + a.apply_error_count,
> + a.sync_error_count,
> + a.stats_reset
> + FROM pg_subscription as s,
> + pg_stat_get_subscription_activity(oid) as a;
>
> inconsistent tab/space indenting for 'a.stats_reset'.
>
> ~~~
>
> 6. src/backend/postmaster/pgstat.c - function name
>
> +/* ----------
> + * pgstat_reset_subscription_counter() -
> + *
> + * Tell the statistics collector to reset a single subscription
> + * counter, or all subscription counters (when subid is InvalidOid).
> + *
> + * Permission checking for this function is managed through the normal
> + * GRANT system.
> + * ----------
> + */
> +void
> +pgstat_reset_subscription_counter(Oid subid)
>
> SUGGESTION (function name)
> "pgstat_reset_subscription_counter" -->
> "pgstat_reset_subscription_counters" (plural)
Fixed.
>
> ~~
>
> 7. src/backend/postmaster/pgstat.c - pgstat_recv_resetsubcounter
>
> @@ -5645,6 +5598,51 @@
> pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg,
> }
> }
>
> +/* ----------
> + * pgstat_recv_resetsubcounter() -
> + *
> + * Reset some subscription statistics of the cluster.
> + * ----------
> + */
> +static void
> +pgstat_recv_resetsubcounter(PgStat_MsgResetsubcounter *msg, int len)
>
>
> "Reset some" seems a bit vague. Why not describe that it is all or
> none according to the msg->m_subid?
I think it reset none, one, or all statistics, actually. Given other
pgstat_recv_reset* functions also have similar comments, I think we
can use it rather than elaborating.
>
> ~~~
>
> 8. src/backend/postmaster/pgstat.c - pgstat_recv_resetsubcounter
>
> + if (!OidIsValid(msg->m_subid))
> + {
> + HASH_SEQ_STATUS sstat;
> +
> + /* Clear all subscription counters */
> + hash_seq_init(&sstat, subscriptionStatHash);
> + while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&sstat)) != NULL)
> + pgstat_reset_subscription(subentry, ts);
> + }
> + else
> + {
> + /* Get the subscription statistics to reset */
> + subentry = pgstat_get_subscription_entry(msg->m_subid, false);
> +
> + /*
> + * Nothing to do if the given subscription entry is not found. This
> + * could happen when the subscription with the subid is removed and
> + * the corresponding statistics entry is also removed before receiving
> + * the reset message.
> + */
> + if (!subentry)
> + return;
> +
> + /* Reset the stats for the requested replication slot */
> + pgstat_reset_subscription(subentry, ts);
> + }
> +}
>
> Why not reverse the if/else?
>
> Checking OidIsValid(...) seems more natural than checking !OidIsValid(...)
Yes, but it's because we use the same pattern in the near function
(see pgstat_recv_resetreplslotcounter()).
>
> ~~~
>
> 9. src/backend/postmaster/pgstat.c - pgstat_recv_subscription_purge
>
> static void
> pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len)
> {
> /* Return if we don't have replication subscription statistics */
> if (subscriptionStatHash == NULL)
> return;
>
> /* Remove from hashtable if present; we don't care if it's not */
> (void) hash_search(subscriptionStatHash, (void *) &(msg->m_subid),
> HASH_REMOVE, NULL);
> }
>
> SUGGESTION
> Wouldn't the above code be simpler written like:
>
> if (subscriptionStatHash)
> {
> /* Remove from hashtable if present; we don't care if it's not */
> (void) hash_search(subscriptionStatHash, (void *) &(msg->m_subid),
> HASH_REMOVE, NULL);
> }
Similarly, as Amit also mentioned, there is a similar pattern in the
near function. So keep it as it is
> ~~~
>
> 10. src/backend/replication/logical/worker.c
>
> (from my previous [Peter-v1] #9)
>
> >> + /* Report the worker failed during table synchronization */
> >> + pgstat_report_subscription_error(MyLogicalRepWorker->subid, false);
> >>
> >> and
> >>
> >> + /* Report the worker failed during the application of the change */
> >> + pgstat_report_subscription_error(MyLogicalRepWorker->subid, true);
> >>
> >>
> >> Why don't these use MySubscription->oid instead of MyLogicalRepWorker->subid?
>
> > It's just because we used to use MyLogicalRepWorker->subid, is there
> > any particular reason why we should use MySubscription->oid here?
>
> I felt MySubscription->oid is a more natural and more direct way of
> expressing the same thing.
>
> Consider: "the oid of the current subscription" versus "the oid of
> the subscription of the current worker". IMO the first one is simpler.
> YMMV.
>
> Also, it is shorter :)
Changed.
>
> ~~~
>
> 11. src/include/pgstat.h - enum order
>
> (follow-on from my previous v1 review comment #10)
>
> >I assume you're concerned about binary compatibility or something. I
> >think it should not be a problem since both
> >PGSTAT_MTYPE_SUBWORKERERROR and PGSTAT_MTYPE_SUBSCRIPTIONPURGE are
> >introduced to PG15.
>
> Yes, maybe it is OK for those ones. But now in v2 there is a new
> PGSTAT_MTYPE_RESETSUBCOUNTER.
>
> Shouldn't at least that one be put at the end for the same reason?
I think it's better to put it near similar RESET enums.
>
> ~~~
>
> 12. src/include/pgstat.h - PgStat_MsgResetsubcounter
>
> Maybe a better name for this is "PgStat_MsgResetsubcounters" (plural)?
I think it's better to be consistent with other similar message
structs (e.g., msg_resetsharedcounter and msg_resetslrucounter).
>
> ~~~
>
> 14. src/tools/pgindent/typedefs.list
>
> PgStat_MsgResetsubcounter (from pgstat.h) is missing?
>
Added.
> ------
>
> 15. pg_stat_subscription_activity view name?
>
> Has the view name already been decided or still under discussion - I
> was not sure?
>
> If is it already decided then fine, but if not then my vote would be
> for something different like:
> e.g.1 - pg_stat_subscription_errors
> e.g.2 - pg_stat_subscription_counters
> e.g.3 - pg_stat_subscription_metrics
>
> Maybe "activity" was chosen to be deliberately vague in case some
> future unknown stats columns get added? But it means now there is a
> corresponding function "pg_stat_reset_subscription_activity", when in
> practice you don't really reset activity - what you want to do is
> reset some statistics *about* the activity... so it all seems a bit
> odd to me.
Yes, it still needs discussion.
I've attached the updated patch.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
Attachments:
[application/octet-stream] v3-0001-Reconsider-pg_stat_subscription_workers-view.patch (71.8K, ../../CAD21AoB_yY6veu5h4mgdCzH64KY4Y9Ln2L2r7wmurV1+y+ig4w@mail.gmail.com/2-v3-0001-Reconsider-pg_stat_subscription_workers-view.patch)
download | inline diff:
From 12fae92fadb6b65c6ccd8b487d8c4f6569ab5828 Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <[email protected]>
Date: Tue, 22 Feb 2022 10:21:53 +0900
Subject: [PATCH v3] Reconsider pg_stat_subscription_workers view.
It was decided (refer to the Discussion link below) that the stats
collector is not an appropriate place to store the error information of
subscription workers.
This patch changes the pg_stat_subscription_workers view (introduced
by commit 8d74fc9) so that it stores only statistics counters:
apply_error_count and sync_error_count, and has one entry for
each subscription. The removed error information such as error-XID and
the error message would be stored in another way in the future in
which is more reliable and persistent.
After removing these error details, there are no longer any relation
information, so the subscription statistics are now a cluster-wide
statistics.
The patch also changes the view name to pg_stat_subscription_activity
since the word "worker" is an implementation detail that we use one
worker for one tablesync and one apply.
Discussion: https://postgr.es/m/[email protected]
---
doc/src/sgml/logical-replication.sgml | 4 +-
doc/src/sgml/monitoring.sgml | 100 +---
src/backend/catalog/system_functions.sql | 4 +-
src/backend/catalog/system_views.sql | 27 +-
src/backend/postmaster/pgstat.c | 583 +++++++++-----------
src/backend/replication/logical/worker.c | 44 +-
src/backend/utils/adt/pgstatfuncs.c | 156 ++----
src/include/catalog/pg_proc.dat | 27 +-
src/include/pgstat.h | 116 ++--
src/test/regress/expected/rules.out | 23 +-
src/test/subscription/t/026_stats.pl | 102 ++++
src/test/subscription/t/026_worker_stats.pl | 165 ------
src/tools/pgindent/typedefs.list | 6 +-
13 files changed, 539 insertions(+), 818 deletions(-)
create mode 100644 src/test/subscription/t/026_stats.pl
delete mode 100644 src/test/subscription/t/026_worker_stats.pl
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 96b4886e08..fb4472356d 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -346,9 +346,7 @@
<para>
A conflict will produce an error and will stop the replication; it must be
resolved manually by the user. Details about the conflict can be found in
- <link linkend="monitoring-pg-stat-subscription-workers">
- <structname>pg_stat_subscription_workers</structname></link> and the
- subscriber's server log.
+ the subscriber's server log.
</para>
<para>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index bf7625d988..55b150dad9 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -628,11 +628,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</row>
<row>
- <entry><structname>pg_stat_subscription_workers</structname><indexterm><primary>pg_stat_subscription_workers</primary></indexterm></entry>
- <entry>One row per subscription worker, showing statistics about errors
- that occurred on that subscription worker.
- See <link linkend="monitoring-pg-stat-subscription-workers">
- <structname>pg_stat_subscription_workers</structname></link> for details.
+ <entry><structname>pg_stat_subscription_activity</structname><indexterm><primary>pg_stat_subscription_activity</primary></indexterm></entry>
+ <entry>One row per subscription, showing statistics about subscription
+ activity.
+ See <link linkend="monitoring-pg-stat-subscription-activity">
+ <structname>pg_stat_subscription_activity</structname></link> for details.
</entry>
</row>
@@ -3063,23 +3063,20 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
</sect2>
- <sect2 id="monitoring-pg-stat-subscription-workers">
- <title><structname>pg_stat_subscription_workers</structname></title>
+ <sect2 id="monitoring-pg-stat-subscription-activity">
+ <title><structname>pg_stat_subscription_activity</structname></title>
<indexterm>
- <primary>pg_stat_subscription_workers</primary>
+ <primary>pg_stat_subscription_activity</primary>
</indexterm>
<para>
- The <structname>pg_stat_subscription_workers</structname> view will contain
- one row per subscription worker on which errors have occurred, for workers
- applying logical replication changes and workers handling the initial data
- copy of the subscribed tables. The statistics entry is removed when the
- corresponding subscription is dropped.
+ The <structname>pg_stat_subscription_activity</structname> view will contain
+ one row per subscription.
</para>
- <table id="pg-stat-subscription-workers" xreflabel="pg_stat_subscription_workers">
- <title><structname>pg_stat_subscription_workers</structname> View</title>
+ <table id="pg-stat-subscription-activity" xreflabel="pg_stat_subscription_activity">
+ <title><structname>pg_stat_subscription_activity</structname> View</title>
<tgroup cols="1">
<thead>
<row>
@@ -3113,72 +3110,31 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>subrelid</structfield> <type>oid</type>
+ <structfield>apply_error_count</structfield> <type>bigint</type>
</para>
<para>
- OID of the relation that the worker is synchronizing; null for the
- main apply worker
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_relid</structfield> <type>oid</type>
- </para>
- <para>
- OID of the relation that the worker was processing when the
- error occurred
+ Number of times an error occurred during the application of changes
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_command</structfield> <type>text</type>
+ <structfield>sync_error_count</structfield> <type>bigint</type>
</para>
<para>
- Name of command being applied when the error occurred. This field
- is null if the error was reported during the initial data copy.
+ Number of times an error occurred during the initial table
+ synchronization
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_xid</structfield> <type>xid</type>
- </para>
- <para>
- Transaction ID of the publisher node being applied when the error
- occurred. This field is null if the error was reported
- during the initial data copy.
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_count</structfield> <type>uint8</type>
- </para>
- <para>
- Number of consecutive times the error occurred
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_message</structfield> <type>text</type>
- </para>
- <para>
- The error message
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_time</structfield> <type>timestamp with time zone</type>
+ <structfield>stats_reset</structfield> <type>timestamp with time zone</type>
</para>
<para>
- Last time at which this error occurred
+ Time at which these statistics were last reset
</para></entry>
</row>
-
</tbody>
</tgroup>
</table>
@@ -5320,22 +5276,16 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
- <primary>pg_stat_reset_subscription_worker</primary>
+ <primary>pg_stat_reset_subscription_activity</primary>
</indexterm>
- <function>pg_stat_reset_subscription_worker</function> ( <parameter>subid</parameter> <type>oid</type> <optional>, <parameter>relid</parameter> <type>oid</type> </optional> )
+ <function>pg_stat_reset_subscription_activity</function> ( <type>oid</type> )
<returnvalue>void</returnvalue>
</para>
<para>
- Resets the statistics of subscription workers running on the
- subscription with <parameter>subid</parameter> shown in the
- <structname>pg_stat_subscription_workers</structname> view. If the
- argument <parameter>relid</parameter> is not <literal>NULL</literal>,
- resets statistics of the subscription worker handling the initial data
- copy of the relation with <parameter>relid</parameter>. Otherwise,
- resets the subscription worker statistics of the main apply worker.
- If the argument <parameter>relid</parameter> is omitted, resets the
- statistics of all subscription workers running on the subscription
- with <parameter>subid</parameter>.
+ Resets statistics for a single subscription shown in the
+ <structname>pg_stat_subscription_activity</structname> view to zero. If
+ the argument is <literal>NULL</literal>, reset statistics for all
+ subscriptions.
</para>
<para>
This function is restricted to superusers by default, but other users
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index fd1421788e..a2f6b15d2a 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -639,9 +639,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_reset_single_function_counters(oid) FROM publ
REVOKE EXECUTE ON FUNCTION pg_stat_reset_replication_slot(text) FROM public;
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid) FROM public;
-
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid, oid) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_activity(oid) FROM public;
REVOKE EXECUTE ON FUNCTION lo_import(text) FROM public;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 3cb69b1f87..0fa5d2a24e 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1264,25 +1264,12 @@ GRANT SELECT (oid, subdbid, subname, subowner, subenabled, subbinary,
substream, subtwophasestate, subslotname, subsynccommit, subpublications)
ON pg_subscription TO public;
-CREATE VIEW pg_stat_subscription_workers AS
+CREATE VIEW pg_stat_subscription_activity AS
SELECT
- w.subid,
+ a.subid,
s.subname,
- w.subrelid,
- w.last_error_relid,
- w.last_error_command,
- w.last_error_xid,
- w.last_error_count,
- w.last_error_message,
- w.last_error_time
- FROM (SELECT
- oid as subid,
- NULL as relid
- FROM pg_subscription
- UNION ALL
- SELECT
- srsubid as subid,
- srrelid as relid
- FROM pg_subscription_rel) sr,
- LATERAL pg_stat_get_subscription_worker(sr.subid, sr.relid) w
- JOIN pg_subscription s ON (w.subid = s.oid);
+ a.apply_error_count,
+ a.sync_error_count,
+ a.stats_reset
+ FROM pg_subscription as s,
+ pg_stat_get_subscription_activity(oid) as a;
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 0646f53098..a5df102e63 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -106,7 +106,7 @@
#define PGSTAT_DB_HASH_SIZE 16
#define PGSTAT_TAB_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
-#define PGSTAT_SUBWORKER_HASH_SIZE 32
+#define PGSTAT_SUBSCRIPTION_HASH_SIZE 32
#define PGSTAT_REPLSLOT_HASH_SIZE 32
@@ -284,6 +284,7 @@ static PgStat_GlobalStats globalStats;
static PgStat_WalStats walStats;
static PgStat_SLRUStats slruStats[SLRU_NUM_ELEMENTS];
static HTAB *replSlotStatHash = NULL;
+static HTAB *subscriptionStatHash = NULL;
/*
* List of OIDs of databases we need to write out. If an entry is InvalidOid,
@@ -322,14 +323,13 @@ NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_no
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
Oid tableoid, bool create);
-static PgStat_StatSubWorkerEntry *pgstat_get_subworker_entry(PgStat_StatDBEntry *dbentry,
- Oid subid, Oid subrelid,
- bool create);
+static PgStat_StatSubEntry *pgstat_get_subscription_entry(Oid subid, bool create);
+static void pgstat_reset_subscription(PgStat_StatSubEntry *subentry, TimestampTz ts);
static void pgstat_write_statsfiles(bool permanent, bool allDbs);
static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- HTAB *subworkerhash, bool permanent);
+ bool permanent);
static void backend_read_statsfile(void);
static bool pgstat_write_statsfile_needed(void);
@@ -341,7 +341,6 @@ static void pgstat_reset_replslot(PgStat_StatReplSlotEntry *slotstats, Timestamp
static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg, TimestampTz now);
static void pgstat_send_funcstats(void);
static void pgstat_send_slru(void);
-static void pgstat_send_subscription_purge(PgStat_MsgSubscriptionPurge *msg);
static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
static bool pgstat_should_report_connstat(void);
static void pgstat_report_disconnect(Oid dboid);
@@ -363,6 +362,7 @@ static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, in
static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
static void pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len);
static void pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg, int len);
+static void pgstat_recv_resetsubcounter(PgStat_MsgResetsubcounter *msg, int len);
static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
@@ -381,7 +381,7 @@ static void pgstat_recv_disconnect(PgStat_MsgDisconnect *msg, int len);
static void pgstat_recv_replslot(PgStat_MsgReplSlot *msg, int len);
static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
static void pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len);
-static void pgstat_recv_subworker_error(PgStat_MsgSubWorkerError *msg, int len);
+static void pgstat_recv_subscription_error(PgStat_MsgSubscriptionError *msg, int len);
/* ------------------------------------------------------------
* Public functions called from postmaster follow
@@ -1187,6 +1187,32 @@ pgstat_vacuum_stat(void)
}
}
+ /*
+ * Repeat the above steps for subscriptions, if subscription stats are
+ * being collected.
+ */
+ if (subscriptionStatHash)
+ {
+ PgStat_StatSubEntry *subentry;
+
+ /*
+ * Read pg_subscription and make a list of OIDs of all existing
+ * subscriptions
+ */
+ htab = pgstat_collect_oids(SubscriptionRelationId, Anum_pg_subscription_oid);
+
+ hash_seq_init(&hstat, subscriptionStatHash);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&hstat)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (hash_search(htab, (void *) &(subentry->subid), HASH_FIND, NULL) == NULL)
+ pgstat_report_subscription_drop(subentry->subid);
+ }
+
+ hash_destroy(htab);
+ }
+
/*
* Lookup our own database entry; if not found, nothing more to do.
*/
@@ -1311,74 +1337,6 @@ pgstat_vacuum_stat(void)
hash_destroy(htab);
}
-
- /*
- * Repeat for subscription workers. Similarly, we needn't bother in the
- * common case where no subscription workers' stats are being collected.
- */
- if (dbentry->subworkers != NULL &&
- hash_get_num_entries(dbentry->subworkers) > 0)
- {
- PgStat_StatSubWorkerEntry *subwentry;
- PgStat_MsgSubscriptionPurge spmsg;
-
- /*
- * Read pg_subscription and make a list of OIDs of all existing
- * subscriptions
- */
- htab = pgstat_collect_oids(SubscriptionRelationId, Anum_pg_subscription_oid);
-
- spmsg.m_databaseid = MyDatabaseId;
- spmsg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&hstat)) != NULL)
- {
- bool exists = false;
- Oid subid = subwentry->key.subid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &subid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * It is possible that we have multiple entries for the
- * subscription corresponding to apply worker and tablesync
- * workers. In such cases, we don't need to add the same subid
- * again.
- */
- for (int i = 0; i < spmsg.m_nentries; i++)
- {
- if (spmsg.m_subids[i] == subid)
- {
- exists = true;
- break;
- }
- }
-
- if (exists)
- continue;
-
- /* This subscription is dead, add the subid to the message */
- spmsg.m_subids[spmsg.m_nentries++] = subid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (spmsg.m_nentries >= PGSTAT_NUM_SUBSCRIPTIONPURGE)
- {
- pgstat_send_subscription_purge(&spmsg);
- spmsg.m_nentries = 0;
- }
- }
-
- /* Send the rest of dead subscriptions */
- if (spmsg.m_nentries > 0)
- pgstat_send_subscription_purge(&spmsg);
-
- hash_destroy(htab);
- }
}
@@ -1551,8 +1509,7 @@ pgstat_reset_shared_counters(const char *target)
* ----------
*/
void
-pgstat_reset_single_counter(Oid objoid, Oid subobjoid,
- PgStat_Single_Reset_Type type)
+pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
PgStat_MsgResetsinglecounter msg;
@@ -1563,7 +1520,6 @@ pgstat_reset_single_counter(Oid objoid, Oid subobjoid,
msg.m_databaseid = MyDatabaseId;
msg.m_resettype = type;
msg.m_objectid = objoid;
- msg.m_subobjectid = subobjoid;
pgstat_send(&msg, sizeof(msg));
}
@@ -1623,6 +1579,30 @@ pgstat_reset_replslot_counter(const char *name)
pgstat_send(&msg, sizeof(msg));
}
+/* ----------
+ * pgstat_reset_subscription_counters() -
+ *
+ * Tell the statistics collector to reset a single subscription
+ * counter, or all subscription counters (when subid is InvalidOid).
+ *
+ * Permission checking for this function is managed through the normal
+ * GRANT system.
+ * ----------
+ */
+void
+pgstat_reset_subscription_counters(Oid subid)
+{
+ PgStat_MsgResetsubcounter msg;
+
+ if (pgStatSock == PGINVALID_SOCKET)
+ return;
+
+ msg.m_subid = subid;
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSUBCOUNTER);
+
+ pgstat_send(&msg, sizeof(msg));
+}
+
/* ----------
* pgstat_report_autovac() -
*
@@ -1949,31 +1929,21 @@ pgstat_report_replslot_drop(const char *slotname)
}
/* ----------
- * pgstat_report_subworker_error() -
+ * pgstat_report_subscription_error() -
*
- * Tell the collector about the subscription worker error.
+ * Tell the collector about the subscription error.
* ----------
*/
void
-pgstat_report_subworker_error(Oid subid, Oid subrelid, Oid relid,
- LogicalRepMsgType command, TransactionId xid,
- const char *errmsg)
+pgstat_report_subscription_error(Oid subid, bool is_apply_error)
{
- PgStat_MsgSubWorkerError msg;
- int len;
+ PgStat_MsgSubscriptionError msg;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBWORKERERROR);
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBSCRIPTIONERROR);
msg.m_databaseid = MyDatabaseId;
msg.m_subid = subid;
- msg.m_subrelid = subrelid;
- msg.m_relid = relid;
- msg.m_command = command;
- msg.m_xid = xid;
- msg.m_timestamp = GetCurrentTimestamp();
- strlcpy(msg.m_message, errmsg, PGSTAT_SUBWORKERERROR_MSGLEN);
-
- len = offsetof(PgStat_MsgSubWorkerError, m_message) + strlen(msg.m_message) + 1;
- pgstat_send(&msg, len);
+ msg.m_is_apply_error = is_apply_error;
+ pgstat_send(&msg, sizeof(PgStat_MsgSubscriptionError));
}
/* ----------
@@ -1987,10 +1957,9 @@ pgstat_report_subscription_drop(Oid subid)
{
PgStat_MsgSubscriptionPurge msg;
- msg.m_databaseid = MyDatabaseId;
- msg.m_subids[0] = subid;
- msg.m_nentries = 1;
- pgstat_send_subscription_purge(&msg);
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBSCRIPTIONPURGE);
+ msg.m_subid = subid;
+ pgstat_send(&msg, sizeof(PgStat_MsgSubscriptionPurge));
}
/* ----------
@@ -3000,32 +2969,19 @@ pgstat_fetch_stat_funcentry(Oid func_id)
/*
* ---------
- * pgstat_fetch_stat_subworker_entry() -
+ * pgstat_fetch_stat_subscription() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for subscription worker or NULL.
+ * the collected statistics for one subscription or NULL.
* ---------
*/
-PgStat_StatSubWorkerEntry *
-pgstat_fetch_stat_subworker_entry(Oid subid, Oid subrelid)
+PgStat_StatSubEntry *
+pgstat_fetch_stat_subscription(Oid subid)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *wentry = NULL;
-
/* Load the stats file if needed */
backend_read_statsfile();
- /*
- * Lookup our database, then find the requested subscription worker stats.
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->subworkers != NULL)
- {
- wentry = pgstat_get_subworker_entry(dbentry, subid, subrelid,
- false);
- }
-
- return wentry;
+ return pgstat_get_subscription_entry(subid, false);
}
/*
@@ -3465,24 +3421,6 @@ pgstat_send_slru(void)
}
}
-/* --------
- * pgstat_send_subscription_purge() -
- *
- * Send a subscription purge message to the collector
- * --------
- */
-static void
-pgstat_send_subscription_purge(PgStat_MsgSubscriptionPurge *msg)
-{
- int len;
-
- len = offsetof(PgStat_MsgSubscriptionPurge, m_subids[0])
- + msg->m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg->m_hdr, PGSTAT_MTYPE_SUBSCRIPTIONPURGE);
- pgstat_send(msg, len);
-}
-
/* ----------
* PgstatCollectorMain() -
*
@@ -3668,6 +3606,10 @@ PgstatCollectorMain(int argc, char *argv[])
len);
break;
+ case PGSTAT_MTYPE_RESETSUBCOUNTER:
+ pgstat_recv_resetsubcounter(&msg.msg_resetsubcounter, len);
+ break;
+
case PGSTAT_MTYPE_AUTOVAC_START:
pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
break;
@@ -3742,8 +3684,8 @@ PgstatCollectorMain(int argc, char *argv[])
pgstat_recv_subscription_purge(&msg.msg_subscriptionpurge, len);
break;
- case PGSTAT_MTYPE_SUBWORKERERROR:
- pgstat_recv_subworker_error(&msg.msg_subworkererror, len);
+ case PGSTAT_MTYPE_SUBSCRIPTIONERROR:
+ pgstat_recv_subscription_error(&msg.msg_subscriptionerror, len);
break;
default:
@@ -3791,8 +3733,7 @@ PgstatCollectorMain(int argc, char *argv[])
/*
* Subroutine to clear stats in a database entry
*
- * Tables, functions, and subscription workers hashes are initialized
- * to empty.
+ * Tables and functions hashes are initialized to empty.
*/
static void
reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
@@ -3845,13 +3786,6 @@ reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
PGSTAT_FUNCTION_HASH_SIZE,
&hash_ctl,
HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(PgStat_StatSubWorkerKey);
- hash_ctl.entrysize = sizeof(PgStat_StatSubWorkerEntry);
- dbentry->subworkers = hash_create("Per-database subscription worker",
- PGSTAT_SUBWORKER_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
}
/*
@@ -3876,7 +3810,7 @@ pgstat_get_db_entry(Oid databaseid, bool create)
/*
* If not found, initialize the new one. This creates empty hash tables
- * for tables, functions, and subscription workers, too.
+ * for tables and functions, too.
*/
if (!found)
reset_dbentry_counters(result);
@@ -3935,45 +3869,65 @@ pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
}
/* ----------
- * pgstat_get_subworker_entry
+ * pgstat_get_subscription_entry
*
- * Return subscription worker entry with the given subscription OID and
- * relation OID. If subrelid is InvalidOid, it returns an entry of the
- * apply worker otherwise returns an entry of the table sync worker
- * associated with subrelid. If no subscription worker entry exists,
- * initialize it, if the create parameter is true. Else, return NULL.
+ * Return the subscription statistics entry with the given subscription OID.
+ * If no subscription entry exists, initialize it, if the create parameter is
+ * true. Else, return NULL.
* ----------
*/
-static PgStat_StatSubWorkerEntry *
-pgstat_get_subworker_entry(PgStat_StatDBEntry *dbentry, Oid subid, Oid subrelid,
- bool create)
+static PgStat_StatSubEntry *
+pgstat_get_subscription_entry(Oid subid, bool create)
{
- PgStat_StatSubWorkerEntry *subwentry;
- PgStat_StatSubWorkerKey key;
+ PgStat_StatSubEntry *subentry;
bool found;
HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
- key.subid = subid;
- key.subrelid = subrelid;
- subwentry = (PgStat_StatSubWorkerEntry *) hash_search(dbentry->subworkers,
- (void *) &key,
- action, &found);
+ if (subscriptionStatHash == NULL)
+ {
+ HASHCTL hash_ctl;
+
+ /*
+ * Quick return NULL if the hash table is empty and the caller didn't
+ * request to create the entry.
+ */
+ if (!create)
+ return NULL;
+
+ hash_ctl.keysize = sizeof(Oid);
+ hash_ctl.entrysize = sizeof(PgStat_StatSubEntry);
+ subscriptionStatHash = hash_create("Subscription hash",
+ PGSTAT_SUBSCRIPTION_HASH_SIZE,
+ &hash_ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ subentry = (PgStat_StatSubEntry *) hash_search(subscriptionStatHash,
+ (void *) &subid,
+ action, &found);
if (!create && !found)
return NULL;
/* If not found, initialize the new one */
if (!found)
- {
- subwentry->last_error_relid = InvalidOid;
- subwentry->last_error_command = 0;
- subwentry->last_error_xid = InvalidTransactionId;
- subwentry->last_error_count = 0;
- subwentry->last_error_time = 0;
- subwentry->last_error_message[0] = '\0';
- }
+ pgstat_reset_subscription(subentry, 0);
- return subwentry;
+ return subentry;
+}
+
+/* ----------
+ * pgstat_reset_subscription
+ *
+ * Reset the given subscription stats.
+ * ----------
+ */
+static void
+pgstat_reset_subscription(PgStat_StatSubEntry *subentry, TimestampTz ts)
+{
+ subentry->apply_error_count = 0;
+ subentry->sync_error_count = 0;
+ subentry->stat_reset_timestamp = ts;
}
/* ----------
@@ -4059,8 +4013,8 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
{
/*
- * Write out the table, function, and subscription-worker stats for
- * this DB into the appropriate per-DB stat file, if required.
+ * Write out the table and function stats for this DB into the
+ * appropriate per-DB stat file, if required.
*/
if (allDbs || pgstat_db_requested(dbentry->databaseid))
{
@@ -4095,6 +4049,22 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
}
}
+ /*
+ * Write subscription stats struct
+ */
+ if (subscriptionStatHash)
+ {
+ PgStat_StatSubEntry *subentry;
+
+ hash_seq_init(&hstat, subscriptionStatHash);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&hstat)) != NULL)
+ {
+ fputc('S', fpout);
+ rc = fwrite(subentry, sizeof(PgStat_StatSubEntry), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+ }
+
/*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
@@ -4174,10 +4144,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
{
HASH_SEQ_STATUS tstat;
HASH_SEQ_STATUS fstat;
- HASH_SEQ_STATUS sstat;
PgStat_StatTabEntry *tabentry;
PgStat_StatFuncEntry *funcentry;
- PgStat_StatSubWorkerEntry *subwentry;
FILE *fpout;
int32 format_id;
Oid dbid = dbentry->databaseid;
@@ -4232,17 +4200,6 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
(void) rc; /* we'll check for error with ferror */
}
- /*
- * Walk through the database's subscription worker stats table.
- */
- hash_seq_init(&sstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&sstat)) != NULL)
- {
- fputc('S', fpout);
- rc = fwrite(subwentry, sizeof(PgStat_StatSubWorkerEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
/*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
@@ -4301,9 +4258,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
* files after reading; the in-memory status is now authoritative, and the
* files would be out of date in case somebody else reads them.
*
- * If a 'deep' read is requested, table/function/subscription-worker stats are
- * read, otherwise the table/function/subscription-worker hash tables remain
- * empty.
+ * If a 'deep' read is requested, table/function stats are read, otherwise
+ * the table/function hash tables remain empty.
* ----------
*/
static HTAB *
@@ -4482,7 +4438,6 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
dbentry->tables = NULL;
dbentry->functions = NULL;
- dbentry->subworkers = NULL;
/*
* In the collector, disregard the timestamp we read from the
@@ -4494,8 +4449,8 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
dbentry->stats_timestamp = 0;
/*
- * Don't create tables/functions/subworkers hashtables for
- * uninteresting databases.
+ * Don't create tables/functions hashtables for uninteresting
+ * databases.
*/
if (onlydb != InvalidOid)
{
@@ -4520,14 +4475,6 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
&hash_ctl,
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
- hash_ctl.keysize = sizeof(PgStat_StatSubWorkerKey);
- hash_ctl.entrysize = sizeof(PgStat_StatSubWorkerEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->subworkers = hash_create("Per-database subscription worker",
- PGSTAT_SUBWORKER_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
/*
* If requested, read the data from the database-specific
* file. Otherwise we just leave the hashtables empty.
@@ -4536,7 +4483,6 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
pgstat_read_db_statsfile(dbentry->databaseid,
dbentry->tables,
dbentry->functions,
- dbentry->subworkers,
permanent);
break;
@@ -4580,6 +4526,45 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
break;
}
+ /*
+ * 'S' A PgStat_StatSubEntry struct describing subscription
+ * statistics.
+ */
+ case 'S':
+ {
+ PgStat_StatSubEntry subbuf;
+ PgStat_StatSubEntry *subentry;
+
+ if (fread(&subbuf, 1, sizeof(PgStat_StatSubEntry),
+ fpin) != sizeof(PgStat_StatSubEntry))
+ {
+ ereport(pgStatRunningInCollector ? LOG : WARNING,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ if (subscriptionStatHash == NULL)
+ {
+ HASHCTL hash_ctl;
+
+ hash_ctl.keysize = sizeof(Oid);
+ hash_ctl.entrysize = sizeof(PgStat_StatSubEntry);
+ hash_ctl.hcxt = pgStatLocalContext;
+ subscriptionStatHash = hash_create("Subscription hash",
+ PGSTAT_SUBSCRIPTION_HASH_SIZE,
+ &hash_ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
+
+ subentry = (PgStat_StatSubEntry *) hash_search(subscriptionStatHash,
+ (void *) &subbuf.subid,
+ HASH_ENTER, NULL);
+
+ memcpy(subentry, &subbuf, sizeof(subbuf));
+ break;
+ }
+
case 'E':
goto done;
@@ -4614,21 +4599,19 @@ done:
* As in pgstat_read_statsfiles, if the permanent file is requested, it is
* removed after reading.
*
- * Note: this code has the ability to skip storing per-table, per-function, or
- * per-subscription-worker data, if NULL is passed for the corresponding hashtable.
- * That's not used at the moment though.
+ * Note: this code has the ability to skip storing per-table or per-function
+ * data, if NULL is passed for the corresponding hashtable. That's not used
+ * at the moment though.
* ----------
*/
static void
pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- HTAB *subworkerhash, bool permanent)
+ bool permanent)
{
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
- PgStat_StatSubWorkerEntry subwbuf;
- PgStat_StatSubWorkerEntry *subwentry;
FILE *fpin;
int32 format_id;
bool found;
@@ -4742,41 +4725,6 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
memcpy(funcentry, &funcbuf, sizeof(funcbuf));
break;
- /*
- * 'S' A PgStat_StatSubWorkerEntry struct describing
- * subscription worker statistics.
- */
- case 'S':
- if (fread(&subwbuf, 1, sizeof(PgStat_StatSubWorkerEntry),
- fpin) != sizeof(PgStat_StatSubWorkerEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if subscription worker data not wanted.
- */
- if (subworkerhash == NULL)
- break;
-
- subwentry = (PgStat_StatSubWorkerEntry *) hash_search(subworkerhash,
- (void *) &subwbuf.key,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(subwentry, &subwbuf, sizeof(subwbuf));
- break;
-
/*
* 'E' The EOF marker of a complete stats file.
*/
@@ -4829,6 +4777,7 @@ pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
PgStat_WalStats myWalStats;
PgStat_SLRUStats mySLRUStats[SLRU_NUM_ELEMENTS];
PgStat_StatReplSlotEntry myReplSlotStats;
+ PgStat_StatSubEntry mySubStats;
FILE *fpin;
int32 format_id;
const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
@@ -4959,6 +4908,22 @@ pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
}
break;
+ /*
+ * 'S' A PgStat_StatSubEntry struct describing subscription
+ * statistics follows.
+ */
+ case 'S':
+ if (fread(&mySubStats, 1, sizeof(PgStat_StatSubEntry), fpin)
+ != sizeof(PgStat_StatSubEntry))
+ {
+ ereport(pgStatRunningInCollector ? LOG : WARNING,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ FreeFile(fpin);
+ return false;
+ }
+ break;
+
case 'E':
goto done;
@@ -5164,6 +5129,7 @@ pgstat_clear_snapshot(void)
pgStatLocalContext = NULL;
pgStatDBHash = NULL;
replSlotStatHash = NULL;
+ subscriptionStatHash = NULL;
/*
* Historically the backend_status.c facilities lived in this file, and
@@ -5450,8 +5416,6 @@ pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
hash_destroy(dbentry->tables);
if (dbentry->functions != NULL)
hash_destroy(dbentry->functions);
- if (dbentry->subworkers != NULL)
- hash_destroy(dbentry->subworkers);
if (hash_search(pgStatDBHash,
(void *) &dbid,
@@ -5489,16 +5453,13 @@ pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
hash_destroy(dbentry->tables);
if (dbentry->functions != NULL)
hash_destroy(dbentry->functions);
- if (dbentry->subworkers != NULL)
- hash_destroy(dbentry->subworkers);
dbentry->tables = NULL;
dbentry->functions = NULL;
- dbentry->subworkers = NULL;
/*
* Reset database-level stats, too. This creates empty hash tables for
- * tables, functions, and subscription workers.
+ * tables and functions.
*/
reset_dbentry_counters(dbentry);
}
@@ -5567,14 +5528,6 @@ pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
else if (msg->m_resettype == RESET_FUNCTION)
(void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_SUBWORKER)
- {
- PgStat_StatSubWorkerKey key;
-
- key.subid = msg->m_objectid;
- key.subrelid = msg->m_subobjectid;
- (void) hash_search(dbentry->subworkers, (void *) &key, HASH_REMOVE, NULL);
- }
}
/* ----------
@@ -5645,6 +5598,51 @@ pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg,
}
}
+/* ----------
+ * pgstat_recv_resetsubcounter() -
+ *
+ * Reset some subscription statistics of the cluster.
+ * ----------
+ */
+static void
+pgstat_recv_resetsubcounter(PgStat_MsgResetsubcounter *msg, int len)
+{
+ PgStat_StatSubEntry *subentry;
+ TimestampTz ts;
+
+ /* Return if we don't have replication subscription statistics */
+ if (subscriptionStatHash == NULL)
+ return;
+
+ ts = GetCurrentTimestamp();
+ if (!OidIsValid(msg->m_subid))
+ {
+ HASH_SEQ_STATUS sstat;
+
+ /* Clear all subscription counters */
+ hash_seq_init(&sstat, subscriptionStatHash);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&sstat)) != NULL)
+ pgstat_reset_subscription(subentry, ts);
+ }
+ else
+ {
+ /* Get the subscription statistics to reset */
+ subentry = pgstat_get_subscription_entry(msg->m_subid, false);
+
+ /*
+ * Nothing to do if the given subscription entry is not found. This
+ * could happen when the subscription with the subid is removed and
+ * the corresponding statistics entry is also removed before receiving
+ * the reset message.
+ */
+ if (!subentry)
+ return;
+
+ /* Reset the stats for the requested replication slot */
+ pgstat_reset_subscription(subentry, ts);
+ }
+}
+
/* ----------
* pgstat_recv_autovac() -
@@ -6126,73 +6124,34 @@ pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
static void
pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *subwentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /* No need to purge if we don't even know the database */
- if (!dbentry || !dbentry->subworkers)
+ /* Return if we don't have replication subscription statistics */
+ if (subscriptionStatHash == NULL)
return;
- /* Remove all subscription worker statistics for the given subscriptions */
- hash_seq_init(&hstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&hstat)) != NULL)
- {
- for (int i = 0; i < msg->m_nentries; i++)
- {
- if (subwentry->key.subid == msg->m_subids[i])
- {
- (void) hash_search(dbentry->subworkers, (void *) &(subwentry->key),
- HASH_REMOVE, NULL);
- break;
- }
- }
- }
+ /* Remove from hashtable if present; we don't care if it's not */
+ (void) hash_search(subscriptionStatHash, (void *) &(msg->m_subid),
+ HASH_REMOVE, NULL);
}
/* ----------
- * pgstat_recv_subworker_error() -
+ * pgstat_recv_subscription_error() -
*
- * Process a SUBWORKERERROR message.
+ * Process a SUBSCRIPTIONERROR message.
* ----------
*/
static void
-pgstat_recv_subworker_error(PgStat_MsgSubWorkerError *msg, int len)
+pgstat_recv_subscription_error(PgStat_MsgSubscriptionError *msg, int len)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *subwentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
+ PgStat_StatSubEntry *subentry;
- /* Get the subscription worker stats */
- subwentry = pgstat_get_subworker_entry(dbentry, msg->m_subid,
- msg->m_subrelid, true);
- Assert(subwentry);
+ /* Get the subscription stats */
+ subentry = pgstat_get_subscription_entry(msg->m_subid, true);
+ Assert(subentry);
- if (subwentry->last_error_relid == msg->m_relid &&
- subwentry->last_error_command == msg->m_command &&
- subwentry->last_error_xid == msg->m_xid &&
- strcmp(subwentry->last_error_message, msg->m_message) == 0)
- {
- /*
- * The same error occurred again in succession, just update its
- * timestamp and count.
- */
- subwentry->last_error_count++;
- subwentry->last_error_time = msg->m_timestamp;
- return;
- }
-
- /* Otherwise, update the error information */
- subwentry->last_error_relid = msg->m_relid;
- subwentry->last_error_command = msg->m_command;
- subwentry->last_error_xid = msg->m_xid;
- subwentry->last_error_count = 1;
- subwentry->last_error_time = msg->m_timestamp;
- strlcpy(subwentry->last_error_message, msg->m_message,
- PGSTAT_SUBWORKERERROR_MSGLEN);
+ if (msg->m_is_apply_error)
+ subentry->apply_error_count++;
+ else
+ subentry->sync_error_count++;
}
/* ----------
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 5d9acc6173..50880f7653 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -3377,7 +3377,6 @@ void
ApplyWorkerMain(Datum main_arg)
{
int worker_slot = DatumGetInt32(main_arg);
- MemoryContext cctx = CurrentMemoryContext;
MemoryContext oldctx;
char originname[NAMEDATALEN];
XLogRecPtr origin_startpos;
@@ -3485,20 +3484,15 @@ ApplyWorkerMain(Datum main_arg)
}
PG_CATCH();
{
- MemoryContext ecxt = MemoryContextSwitchTo(cctx);
- ErrorData *errdata = CopyErrorData();
-
/*
- * Report the table sync error. There is no corresponding message
- * type for table synchronization.
+ * Abort the current transaction so that we send the stats message
+ * in an idle state.
*/
- pgstat_report_subworker_error(MyLogicalRepWorker->subid,
- MyLogicalRepWorker->relid,
- MyLogicalRepWorker->relid,
- 0, /* message type */
- InvalidTransactionId,
- errdata->message);
- MemoryContextSwitchTo(ecxt);
+ AbortOutOfAnyTransaction();
+
+ /* Report the worker failed during table synchronization */
+ pgstat_report_subscription_error(MySubscription->oid, false);
+
PG_RE_THROW();
}
PG_END_TRY();
@@ -3625,22 +3619,14 @@ ApplyWorkerMain(Datum main_arg)
}
PG_CATCH();
{
- /* report the apply error */
- if (apply_error_callback_arg.command != 0)
- {
- MemoryContext ecxt = MemoryContextSwitchTo(cctx);
- ErrorData *errdata = CopyErrorData();
-
- pgstat_report_subworker_error(MyLogicalRepWorker->subid,
- MyLogicalRepWorker->relid,
- apply_error_callback_arg.rel != NULL
- ? apply_error_callback_arg.rel->localreloid
- : InvalidOid,
- apply_error_callback_arg.command,
- apply_error_callback_arg.remote_xid,
- errdata->message);
- MemoryContextSwitchTo(ecxt);
- }
+ /*
+ * Abort the current transaction so that we send the stats message in
+ * an idle state.
+ */
+ AbortOutOfAnyTransaction();
+
+ /* Report the worker failed during the application of the change */
+ pgstat_report_subscription_error(MySubscription->oid, !am_tablesync_worker());
PG_RE_THROW();
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 30e8dfa7c1..466024e828 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2163,7 +2163,7 @@ pg_stat_reset_single_table_counters(PG_FUNCTION_ARGS)
{
Oid taboid = PG_GETARG_OID(0);
- pgstat_reset_single_counter(taboid, InvalidOid, RESET_TABLE);
+ pgstat_reset_single_counter(taboid, RESET_TABLE);
PG_RETURN_VOID();
}
@@ -2173,38 +2173,11 @@ pg_stat_reset_single_function_counters(PG_FUNCTION_ARGS)
{
Oid funcoid = PG_GETARG_OID(0);
- pgstat_reset_single_counter(funcoid, InvalidOid, RESET_FUNCTION);
+ pgstat_reset_single_counter(funcoid, RESET_FUNCTION);
PG_RETURN_VOID();
}
-Datum
-pg_stat_reset_subscription_worker_subrel(PG_FUNCTION_ARGS)
-{
- Oid subid = PG_GETARG_OID(0);
- Oid relid = PG_ARGISNULL(1) ? InvalidOid : PG_GETARG_OID(1);
-
- pgstat_reset_single_counter(subid, relid, RESET_SUBWORKER);
-
- PG_RETURN_VOID();
-}
-
-/* Reset all subscription worker stats associated with the given subscription */
-Datum
-pg_stat_reset_subscription_worker_sub(PG_FUNCTION_ARGS)
-{
- Oid subid = PG_GETARG_OID(0);
-
- /*
- * Use subscription drop message to remove statistics of all subscription
- * workers.
- */
- pgstat_report_subscription_drop(subid);
-
- PG_RETURN_VOID();
-}
-
-
/* Reset SLRU counters (a specific one or all of them). */
Datum
pg_stat_reset_slru(PG_FUNCTION_ARGS)
@@ -2258,6 +2231,32 @@ pg_stat_reset_replication_slot(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+/* Reset subscription stats (a specific one or all of them) */
+Datum
+pg_stat_reset_subscription_activity(PG_FUNCTION_ARGS)
+{
+ Oid subid;
+
+ if (PG_ARGISNULL(0))
+ {
+ /* Clear all subscription stats */
+ subid = InvalidOid;
+ }
+ else
+ {
+ subid = PG_GETARG_OID(0);
+
+ if (!OidIsValid(subid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid subscription OID %u", subid)));
+ }
+
+ pgstat_reset_subscription_counters(subid);
+
+ PG_RETURN_VOID();
+}
+
Datum
pg_stat_get_archiver(PG_FUNCTION_ARGS)
{
@@ -2400,50 +2399,32 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
}
/*
- * Get the subscription worker statistics for the given subscription
- * (and relation).
+ * Get the subscription activity statistics for the given subscription. If the
+ * subscription statistics is not available, return all-zeros stats.
*/
Datum
-pg_stat_get_subscription_worker(PG_FUNCTION_ARGS)
+pg_stat_get_subscription_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_SUBSCRIPTION_WORKER_COLS 8
+#define PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS 4
Oid subid = PG_GETARG_OID(0);
- Oid subrelid;
TupleDesc tupdesc;
- Datum values[PG_STAT_GET_SUBSCRIPTION_WORKER_COLS];
- bool nulls[PG_STAT_GET_SUBSCRIPTION_WORKER_COLS];
- PgStat_StatSubWorkerEntry *wentry;
- int i;
-
- if (PG_ARGISNULL(1))
- subrelid = InvalidOid;
- else
- subrelid = PG_GETARG_OID(1);
+ Datum values[PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS];
+ bool nulls[PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS];
+ PgStat_StatSubEntry *subentry;
+ PgStat_StatSubEntry allzero;
- /* Get subscription worker stats */
- wentry = pgstat_fetch_stat_subworker_entry(subid, subrelid);
-
- /* Return NULL if there is no worker statistics */
- if (wentry == NULL)
- PG_RETURN_NULL();
+ /* Get subscription stats */
+ subentry = pgstat_fetch_stat_subscription(subid);
/* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_WORKER_COLS);
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_ACTIVITY_COLS);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "subid",
OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subrelid",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 3, "last_error_relid",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 4, "last_error_command",
- TEXTOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 5, "last_error_xid",
- XIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 6, "last_error_count",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "apply_error_count",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 7, "last_error_message",
- TEXTOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 8, "last_error_time",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "sync_error_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
@@ -2451,46 +2432,27 @@ pg_stat_get_subscription_worker(PG_FUNCTION_ARGS)
MemSet(values, 0, sizeof(values));
MemSet(nulls, 0, sizeof(nulls));
- i = 0;
- /* subid */
- values[i++] = ObjectIdGetDatum(subid);
-
- /* subrelid */
- if (OidIsValid(subrelid))
- values[i++] = ObjectIdGetDatum(subrelid);
- else
- nulls[i++] = true;
-
- /* last_error_relid */
- if (OidIsValid(wentry->last_error_relid))
- values[i++] = ObjectIdGetDatum(wentry->last_error_relid);
- else
- nulls[i++] = true;
-
- /* last_error_command */
- if (wentry->last_error_command != 0)
- values[i++] =
- CStringGetTextDatum(logicalrep_message_type(wentry->last_error_command));
- else
- nulls[i++] = true;
+ if (!subentry)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(PgStat_StatSubEntry));
+ subentry = &allzero;
+ }
- /* last_error_xid */
- if (TransactionIdIsValid(wentry->last_error_xid))
- values[i++] = TransactionIdGetDatum(wentry->last_error_xid);
- else
- nulls[i++] = true;
+ /* subid */
+ values[0] = ObjectIdGetDatum(subid);
- /* last_error_count */
- values[i++] = Int64GetDatum(wentry->last_error_count);
+ /* apply_error_count */
+ values[1] = Int64GetDatum(subentry->apply_error_count);
- /* last_error_message */
- values[i++] = CStringGetTextDatum(wentry->last_error_message);
+ /* sync_error_count */
+ values[2] = Int64GetDatum(subentry->sync_error_count);
- /* last_error_time */
- if (wentry->last_error_time != 0)
- values[i++] = TimestampTzGetDatum(wentry->last_error_time);
+ /* stats_reset */
+ if (subentry->stat_reset_timestamp == 0)
+ nulls[3] = true;
else
- nulls[i++] = true;
+ values[3] = TimestampTzGetDatum(subentry->stat_reset_timestamp);
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7f1ee97f55..811252c072 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5377,14 +5377,14 @@
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
-{ oid => '8523', descr => 'statistics: information about subscription worker',
- proname => 'pg_stat_get_subscription_worker', prorows => '1', proisstrict => 'f',
- proretset => 't', provolatile => 's', proparallel => 'r',
- prorettype => 'record', proargtypes => 'oid oid',
- proallargtypes => '{oid,oid,oid,oid,oid,text,xid,int8,text,timestamptz}',
- proargmodes => '{i,i,o,o,o,o,o,o,o,o}',
- proargnames => '{subid,subrelid,subid,subrelid,last_error_relid,last_error_command,last_error_xid,last_error_count,last_error_message,last_error_time}',
- prosrc => 'pg_stat_get_subscription_worker' },
+{ oid => '8523', descr => 'statistics: information about subscription activity',
+ proname => 'pg_stat_get_subscription_activity', proisstrict => 'f',
+ provolatile => 's', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o}',
+ proargnames => '{subid,subid,apply_error_count,sync_error_count,stats_reset}',
+ prosrc => 'pg_stat_get_subscription_activity' },
{ oid => '6118', descr => 'statistics: information about subscription',
proname => 'pg_stat_get_subscription', prorows => '10', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
@@ -5773,15 +5773,10 @@
provolatile => 'v', prorettype => 'void', proargtypes => 'text',
prosrc => 'pg_stat_reset_replication_slot' },
{ oid => '8524',
- descr => 'statistics: reset collected statistics for a single subscription worker',
- proname => 'pg_stat_reset_subscription_worker', proisstrict => 'f',
- provolatile => 'v', prorettype => 'void', proargtypes => 'oid oid',
- prosrc => 'pg_stat_reset_subscription_worker_subrel' },
-{ oid => '8525',
- descr => 'statistics: reset all collected statistics for a single subscription',
- proname => 'pg_stat_reset_subscription_worker',
+ descr => 'statistics: reset collected statistics for a single subscription',
+ proname => 'pg_stat_reset_subscription_activity', proisstrict => 'f',
provolatile => 'v', prorettype => 'void', proargtypes => 'oid',
- prosrc => 'pg_stat_reset_subscription_worker_sub' },
+ prosrc => 'pg_stat_reset_subscription_activity' },
{ oid => '3163', descr => 'current trigger depth',
proname => 'pg_trigger_depth', provolatile => 's', proparallel => 'r',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index e10d20222a..4e1351d349 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -67,6 +67,7 @@ typedef enum StatMsgType
PGSTAT_MTYPE_RESETSINGLECOUNTER,
PGSTAT_MTYPE_RESETSLRUCOUNTER,
PGSTAT_MTYPE_RESETREPLSLOTCOUNTER,
+ PGSTAT_MTYPE_RESETSUBCOUNTER,
PGSTAT_MTYPE_AUTOVAC_START,
PGSTAT_MTYPE_VACUUM,
PGSTAT_MTYPE_ANALYZE,
@@ -85,7 +86,7 @@ typedef enum StatMsgType
PGSTAT_MTYPE_CONNECT,
PGSTAT_MTYPE_DISCONNECT,
PGSTAT_MTYPE_SUBSCRIPTIONPURGE,
- PGSTAT_MTYPE_SUBWORKERERROR,
+ PGSTAT_MTYPE_SUBSCRIPTIONERROR,
} StatMsgType;
/* ----------
@@ -148,8 +149,7 @@ typedef enum PgStat_Shared_Reset_Target
typedef enum PgStat_Single_Reset_Type
{
RESET_TABLE,
- RESET_FUNCTION,
- RESET_SUBWORKER
+ RESET_FUNCTION
} PgStat_Single_Reset_Type;
/* ------------------------------------------------------------
@@ -368,7 +368,6 @@ typedef struct PgStat_MsgResetsinglecounter
Oid m_databaseid;
PgStat_Single_Reset_Type m_resettype;
Oid m_objectid;
- Oid m_subobjectid;
} PgStat_MsgResetsinglecounter;
/* ----------
@@ -394,6 +393,19 @@ typedef struct PgStat_MsgResetreplslotcounter
bool clearall;
} PgStat_MsgResetreplslotcounter;
+/* ----------
+ * PgStat_MsgResetsubcounter Sent by the backend to tell the collector
+ * to reset subscription counter(s)
+ * ----------
+ */
+typedef struct PgStat_MsgResetsubcounter
+{
+ PgStat_MsgHdr m_hdr;
+ Oid m_subid; /* InvalidOid for clearing all subscription
+ * stats */
+} PgStat_MsgResetsubcounter;
+
+
/* ----------
* PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
* that a database is going to be processed
@@ -543,52 +555,28 @@ typedef struct PgStat_MsgReplSlot
/* ----------
* PgStat_MsgSubscriptionPurge Sent by the backend and autovacuum to tell the
- * collector about the dead subscriptions.
+ * collector about the dead subscription.
* ----------
*/
-#define PGSTAT_NUM_SUBSCRIPTIONPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) / sizeof(Oid))
-
typedef struct PgStat_MsgSubscriptionPurge
{
PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_subids[PGSTAT_NUM_SUBSCRIPTIONPURGE];
+ Oid m_subid;
} PgStat_MsgSubscriptionPurge;
/* ----------
- * PgStat_MsgSubWorkerError Sent by the apply worker or the table sync
- * worker to report the error occurred while
- * processing changes.
+ * PgStat_MsgSubscriptionError Sent by the apply worker or the table sync
+ * worker to report an error on the subscription.
* ----------
*/
-#define PGSTAT_SUBWORKERERROR_MSGLEN 256
-typedef struct PgStat_MsgSubWorkerError
+typedef struct PgStat_MsgSubscriptionError
{
PgStat_MsgHdr m_hdr;
- /*
- * m_subid and m_subrelid are used to determine the subscription and the
- * reporter of the error. m_subrelid is InvalidOid if reported by an apply
- * worker otherwise reported by a table sync worker.
- */
Oid m_databaseid;
Oid m_subid;
- Oid m_subrelid;
-
- /*
- * Oid of the table that the reporter was actually processing. m_relid can
- * be InvalidOid if an error occurred during worker applying a
- * non-data-modification message such as RELATION.
- */
- Oid m_relid;
-
- LogicalRepMsgType m_command;
- TransactionId m_xid;
- TimestampTz m_timestamp;
- char m_message[PGSTAT_SUBWORKERERROR_MSGLEN];
-} PgStat_MsgSubWorkerError;
+ bool m_is_apply_error;
+} PgStat_MsgSubscriptionError;
/* ----------
* PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
@@ -750,6 +738,7 @@ typedef union PgStat_Msg
PgStat_MsgResetsinglecounter msg_resetsinglecounter;
PgStat_MsgResetslrucounter msg_resetslrucounter;
PgStat_MsgResetreplslotcounter msg_resetreplslotcounter;
+ PgStat_MsgResetsubcounter msg_resetsubcounter;
PgStat_MsgAutovacStart msg_autovacuum_start;
PgStat_MsgVacuum msg_vacuum;
PgStat_MsgAnalyze msg_analyze;
@@ -767,8 +756,8 @@ typedef union PgStat_Msg
PgStat_MsgReplSlot msg_replslot;
PgStat_MsgConnect msg_connect;
PgStat_MsgDisconnect msg_disconnect;
+ PgStat_MsgSubscriptionError msg_subscriptionerror;
PgStat_MsgSubscriptionPurge msg_subscriptionpurge;
- PgStat_MsgSubWorkerError msg_subworkererror;
} PgStat_Msg;
@@ -823,16 +812,11 @@ typedef struct PgStat_StatDBEntry
TimestampTz stats_timestamp; /* time of db stats file update */
/*
- * tables, functions, and subscription workers must be last in the struct,
- * because we don't write the pointers out to the stats file.
- *
- * subworkers is the hash table of PgStat_StatSubWorkerEntry which stores
- * statistics of logical replication workers: apply worker and table sync
- * worker.
+ * tables and functions must be last in the struct, because we don't write
+ * the pointers out to the stats file.
*/
HTAB *tables;
HTAB *functions;
- HTAB *subworkers;
} PgStat_StatDBEntry;
@@ -989,38 +973,17 @@ typedef struct PgStat_StatReplSlotEntry
TimestampTz stat_reset_timestamp;
} PgStat_StatReplSlotEntry;
-/* The lookup key for subscription worker hash table */
-typedef struct PgStat_StatSubWorkerKey
-{
- Oid subid;
-
- /*
- * Oid of the table for which tablesync worker will copy the initial data.
- * An InvalidOid will be assigned for apply workers.
- */
- Oid subrelid;
-} PgStat_StatSubWorkerKey;
-
/*
- * Logical replication apply worker and table sync worker statistics kept in the
- * stats collector.
+ * Subscription statistics kept in the stats collector.
*/
-typedef struct PgStat_StatSubWorkerEntry
+typedef struct PgStat_StatSubEntry
{
- PgStat_StatSubWorkerKey key; /* hash key (must be first) */
+ Oid subid; /* hash key (must be first) */
- /*
- * Subscription worker error statistics representing an error that
- * occurred during application of changes or the initial table
- * synchronization.
- */
- Oid last_error_relid;
- LogicalRepMsgType last_error_command;
- TransactionId last_error_xid;
- PgStat_Counter last_error_count;
- TimestampTz last_error_time;
- char last_error_message[PGSTAT_SUBWORKERERROR_MSGLEN];
-} PgStat_StatSubWorkerEntry;
+ PgStat_Counter apply_error_count;
+ PgStat_Counter sync_error_count;
+ TimestampTz stat_reset_timestamp;
+} PgStat_StatSubEntry;
/*
* Working state needed to accumulate per-function-call timing statistics.
@@ -1111,10 +1074,10 @@ extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
extern void pgstat_reset_shared_counters(const char *);
-extern void pgstat_reset_single_counter(Oid objectid, Oid subobjectid,
- PgStat_Single_Reset_Type type);
+extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_reset_slru_counter(const char *);
extern void pgstat_reset_replslot_counter(const char *name);
+extern void pgstat_reset_subscription_counters(Oid subid);
extern void pgstat_report_connect(Oid dboid);
extern void pgstat_report_autovac(Oid dboid);
@@ -1131,9 +1094,7 @@ extern void pgstat_report_checksum_failure(void);
extern void pgstat_report_replslot(const PgStat_StatReplSlotEntry *repSlotStat);
extern void pgstat_report_replslot_create(const char *slotname);
extern void pgstat_report_replslot_drop(const char *slotname);
-extern void pgstat_report_subworker_error(Oid subid, Oid subrelid, Oid relid,
- LogicalRepMsgType command,
- TransactionId xid, const char *errmsg);
+extern void pgstat_report_subscription_error(Oid subid, bool is_apply_error);
extern void pgstat_report_subscription_drop(Oid subid);
extern void pgstat_initialize(void);
@@ -1226,8 +1187,7 @@ extern void pgstat_send_wal(bool force);
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
-extern PgStat_StatSubWorkerEntry *pgstat_fetch_stat_subworker_entry(Oid subid,
- Oid subrelid);
+extern PgStat_StatSubEntry *pgstat_fetch_stat_subscription(Oid subid);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_BgWriterStats *pgstat_fetch_stat_bgwriter(void);
extern PgStat_CheckpointerStats *pgstat_fetch_stat_checkpointer(void);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 1420288d67..1fe4a98d0e 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2072,24 +2072,13 @@ pg_stat_subscription| SELECT su.oid AS subid,
st.latest_end_time
FROM (pg_subscription su
LEFT JOIN pg_stat_get_subscription(NULL::oid) st(subid, relid, pid, received_lsn, last_msg_send_time, last_msg_receipt_time, latest_end_lsn, latest_end_time) ON ((st.subid = su.oid)));
-pg_stat_subscription_workers| SELECT w.subid,
+pg_stat_subscription_activity| SELECT a.subid,
s.subname,
- w.subrelid,
- w.last_error_relid,
- w.last_error_command,
- w.last_error_xid,
- w.last_error_count,
- w.last_error_message,
- w.last_error_time
- FROM ( SELECT pg_subscription.oid AS subid,
- NULL::oid AS relid
- FROM pg_subscription
- UNION ALL
- SELECT pg_subscription_rel.srsubid AS subid,
- pg_subscription_rel.srrelid AS relid
- FROM pg_subscription_rel) sr,
- (LATERAL pg_stat_get_subscription_worker(sr.subid, sr.relid) w(subid, subrelid, last_error_relid, last_error_command, last_error_xid, last_error_count, last_error_message, last_error_time)
- JOIN pg_subscription s ON ((w.subid = s.oid)));
+ a.apply_error_count,
+ a.sync_error_count,
+ a.stats_reset
+ FROM pg_subscription s,
+ LATERAL pg_stat_get_subscription_activity(s.oid) a(subid, apply_error_count, sync_error_count, stats_reset);
pg_stat_sys_indexes| SELECT pg_stat_all_indexes.relid,
pg_stat_all_indexes.indexrelid,
pg_stat_all_indexes.schemaname,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
new file mode 100644
index 0000000000..72d0943383
--- /dev/null
+++ b/src/test/subscription/t/026_stats.pl
@@ -0,0 +1,102 @@
+
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+# Tests for subscription stats.
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Create publisher node.
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+# Create subscriber node.
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+# Initial table setup on both publisher and subscriber. On subscriber we
+# create the same tables but with primary keys. Also, insert some data that
+# will conflict with the data replicated from publisher later.
+$node_publisher->safe_psql(
+ 'postgres',
+ qq[
+BEGIN;
+CREATE TABLE test_tab1 (a int);
+INSERT INTO test_tab1 VALUES (1);
+COMMIT;
+]);
+$node_subscriber->safe_psql(
+ 'postgres',
+ qq[
+BEGIN;
+CREATE TABLE test_tab1 (a int primary key);
+INSERT INTO test_tab1 VALUES (1);
+COMMIT;
+]);
+
+# Setup publication.
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION tap_pub FOR TABLE test_tab1;");
+
+# There shouldn't be any subscription errors before starting logical replication.
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(1) FROM pg_stat_subscription_activity");
+is($result, qq(0), 'check no subscription error');
+
+# Create subscription. The tablesync for test_tab1 on tap_sub will enter into
+# infinite error loop due to violating the unique constraint.
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub;"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Wait for the tablesync error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT sync_error_count > 0
+FROM pg_stat_subscription_activity
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for tablesync error";
+
+# Truncate test_tab1 so that tablesync worker can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
+
+# Wait for initial tablesync for test_tab1 to finish.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT count(1) = 1 FROM pg_subscription_rel
+WHERE srrelid = 'test_tab1'::regclass AND srsubstate in ('r', 's')
+]) or die "Timed out while waiting for subscriber to synchronize data";
+
+# Check test_tab1 on the subscriber has one row.
+$result = $node_subscriber->safe_psql('postgres', "SELECT a FROM test_tab1");
+is($result, qq(1), 'check the table has now row');
+
+# Insert data to test_tab1 on the publisher, raising an error on the subscriber
+# due to violation of the unique constraint on test_tab1.
+$node_publisher->safe_psql('postgres', "INSERT INTO test_tab1 VALUES (1)");
+
+# Wait for the apply error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT apply_error_count > 0
+FROM pg_stat_subscription_activity
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for apply error";
+
+# Truncate test_tab1 so that apply worker can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
+
+$node_subscriber->stop('fast');
+$node_publisher->stop('fast');
+
+done_testing();
diff --git a/src/test/subscription/t/026_worker_stats.pl b/src/test/subscription/t/026_worker_stats.pl
deleted file mode 100644
index f72e4766e8..0000000000
--- a/src/test/subscription/t/026_worker_stats.pl
+++ /dev/null
@@ -1,165 +0,0 @@
-
-# Copyright (c) 2021-2022, PostgreSQL Global Development Group
-
-# Tests for subscription error stats.
-use strict;
-use warnings;
-use PostgreSQL::Test::Cluster;
-use PostgreSQL::Test::Utils;
-use Test::More;
-
-# Test if the error reported on pg_stat_subscription_workers view is expected.
-sub test_subscription_error
-{
- my ($node, $relname, $command, $xid, $by_apply_worker, $errmsg_prefix, $msg)
- = @_;
-
- my $check_sql = qq[
-SELECT count(1) > 0
-FROM pg_stat_subscription_workers
-WHERE last_error_relid = '$relname'::regclass
- AND starts_with(last_error_message, '$errmsg_prefix')];
-
- # subrelid
- $check_sql .= $by_apply_worker
- ? qq[ AND subrelid IS NULL]
- : qq[ AND subrelid = '$relname'::regclass];
-
- # last_error_command
- $check_sql .= $command eq ''
- ? qq[ AND last_error_command IS NULL]
- : qq[ AND last_error_command = '$command'];
-
- # last_error_xid
- $check_sql .= $xid eq ''
- ? qq[ AND last_error_xid IS NULL]
- : qq[ AND last_error_xid = '$xid'::xid];
-
- # Wait for the particular error statistics to be reported.
- $node->poll_query_until('postgres', $check_sql,
-) or die "Timed out while waiting for " . $msg;
-}
-
-# Create publisher node.
-my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
-$node_publisher->init(allows_streaming => 'logical');
-$node_publisher->start;
-
-# Create subscriber node.
-my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
-$node_subscriber->init(allows_streaming => 'logical');
-
-# The subscriber will enter an infinite error loop, so we don't want
-# to overflow the server log with error messages.
-$node_subscriber->append_conf('postgresql.conf',
- qq[
-wal_retrieve_retry_interval = 2s
-]);
-$node_subscriber->start;
-
-# Initial table setup on both publisher and subscriber. On subscriber we
-# create the same tables but with primary keys. Also, insert some data that
-# will conflict with the data replicated from publisher later.
-$node_publisher->safe_psql(
- 'postgres',
- qq[
-BEGIN;
-CREATE TABLE test_tab1 (a int);
-CREATE TABLE test_tab2 (a int);
-INSERT INTO test_tab1 VALUES (1);
-INSERT INTO test_tab2 VALUES (1);
-COMMIT;
-]);
-$node_subscriber->safe_psql(
- 'postgres',
- qq[
-BEGIN;
-CREATE TABLE test_tab1 (a int primary key);
-CREATE TABLE test_tab2 (a int primary key);
-INSERT INTO test_tab2 VALUES (1);
-COMMIT;
-]);
-
-# Setup publications.
-my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
-$node_publisher->safe_psql(
- 'postgres',
- "CREATE PUBLICATION tap_pub FOR TABLE test_tab1, test_tab2;");
-
-# There shouldn't be any subscription errors before starting logical replication.
-my $result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(1) FROM pg_stat_subscription_workers");
-is($result, qq(0), 'check no subscription error');
-
-# Create subscription. The table sync for test_tab2 on tap_sub will enter into
-# infinite error loop due to violating the unique constraint.
-$node_subscriber->safe_psql(
- 'postgres',
- "CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub;");
-
-$node_publisher->wait_for_catchup('tap_sub');
-
-# Wait for initial table sync for test_tab1 to finish.
-$node_subscriber->poll_query_until(
- 'postgres',
- qq[
-SELECT count(1) = 1 FROM pg_subscription_rel
-WHERE srrelid = 'test_tab1'::regclass AND srsubstate in ('r', 's')
-]) or die "Timed out while waiting for subscriber to synchronize data";
-
-# Check the initial data.
-$result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(a) FROM test_tab1");
-is($result, q(1), 'check initial data are copied to subscriber');
-
-# Insert more data to test_tab1, raising an error on the subscriber due to
-# violation of the unique constraint on test_tab1.
-my $xid = $node_publisher->safe_psql(
- 'postgres',
- qq[
-BEGIN;
-INSERT INTO test_tab1 VALUES (1);
-SELECT pg_current_xact_id()::xid;
-COMMIT;
-]);
-test_subscription_error($node_subscriber, 'test_tab1', 'INSERT', $xid,
- 1, # check apply worker error
- qq(duplicate key value violates unique constraint),
- 'error reported by the apply worker');
-
-# Check the table sync worker's error in the view.
-test_subscription_error($node_subscriber, 'test_tab2', '', '',
- 0, # check tablesync worker error
- qq(duplicate key value violates unique constraint),
- 'the error reported by the table sync worker');
-
-# Test for resetting subscription worker statistics.
-# Truncate test_tab1 and test_tab2 so that applying changes and table sync can
-# continue, respectively.
-$node_subscriber->safe_psql(
- 'postgres',
- "TRUNCATE test_tab1, test_tab2;");
-
-# Wait for the data to be replicated.
-$node_subscriber->poll_query_until(
- 'postgres',
- "SELECT count(1) > 0 FROM test_tab1");
-$node_subscriber->poll_query_until(
- 'postgres',
- "SELECT count(1) > 0 FROM test_tab2");
-
-# There shouldn't be any errors in the view after dropping the subscription.
-$node_subscriber->safe_psql(
- 'postgres',
- "DROP SUBSCRIPTION tap_sub;");
-$result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(1) FROM pg_stat_subscription_workers");
-is($result, q(0), 'no error after dropping subscription');
-
-$node_subscriber->stop('fast');
-$node_publisher->stop('fast');
-
-done_testing();
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c6b302c7b2..0b95c3b010 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1943,9 +1943,10 @@ PgStat_MsgResetreplslotcounter
PgStat_MsgResetsharedcounter
PgStat_MsgResetsinglecounter
PgStat_MsgResetslrucounter
+PgStat_MsgResetsubcounter
PgStat_MsgSLRU
+PgStat_MsgSubscriptionError
PgStat_MsgSubscriptionPurge
-PgStat_MsgSubWorkerError
PgStat_MsgTabpurge
PgStat_MsgTabstat
PgStat_MsgTempFile
@@ -1957,8 +1958,7 @@ PgStat_Single_Reset_Type
PgStat_StatDBEntry
PgStat_StatFuncEntry
PgStat_StatReplSlotEntry
-PgStat_StatSubWorkerEntry
-PgStat_StatSubWorkerKey
+PgStat_StatSubEntry
PgStat_StatTabEntry
PgStat_SubXactStatus
PgStat_TableCounts
--
2.24.3 (Apple Git-128)
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-25 01:56 Peter Smith <[email protected]>
parent: Masahiko Sawada <[email protected]>
2 siblings, 1 reply; 42+ messages in thread
From: Peter Smith @ 2022-02-25 01:56 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: [email protected] <[email protected]>; Amit Kapila <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Below are my review comments for the v3 patch.
======
1. Commit message
(An earlier review comment [Peter-v2] #2 was only partly fixed)
"there are no longer any relation information" --> "there is no longer
any relation information"
~~~
2. doc/src/sgml/monitoring.sgml
+ <entry><structname>pg_stat_subscription_activity</structname><indexterm><primary>pg_stat_subscription_activity</primary></indexterm></entry>
+ <entry>One row per subscription, showing statistics about subscription
+ activity.
+ See <link linkend="monitoring-pg-stat-subscription-activity">
+ <structname>pg_stat_subscription_activity</structname></link>
for details.
</entry>
</row>
Currently these stats are only about errors. These are not really
statistics about "activity" though. Probably it is better just to
avoid that word altogether?
SUGGESTIONS
e.g.1. "One row per subscription, showing statistics about
subscription activity." --> "One row per subscription, showing
statistics about errors."
e.g.2. "One row per subscription, showing statistics about
subscription activity." --> "One row per subscription, showing
statistics about that subscription."
-----
[Peter-v2] https://www.postgresql.org/message-id/CAHut%2BPv%3DVmXtHmPKp4fg8VDF%2BTQP6xWgL91Jn-hrqg5QObfCZA%40ma...
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-25 05:26 Amit Kapila <[email protected]>
parent: Masahiko Sawada <[email protected]>
2 siblings, 0 replies; 42+ messages in thread
From: Amit Kapila @ 2022-02-25 05:26 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Peter Smith <[email protected]>; [email protected] <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Thu, Feb 24, 2022 at 9:20 PM Masahiko Sawada <[email protected]> wrote:
>
> >
> > 6. src/backend/postmaster/pgstat.c - function name
> >
> > +/* ----------
> > + * pgstat_reset_subscription_counter() -
> > + *
> > + * Tell the statistics collector to reset a single subscription
> > + * counter, or all subscription counters (when subid is InvalidOid).
> > + *
> > + * Permission checking for this function is managed through the normal
> > + * GRANT system.
> > + * ----------
> > + */
> > +void
> > +pgstat_reset_subscription_counter(Oid subid)
> >
> > SUGGESTION (function name)
> > "pgstat_reset_subscription_counter" -->
> > "pgstat_reset_subscription_counters" (plural)
>
> Fixed.
>
We don't use the plural form in other similar cases like
pgstat_reset_replslot_counter, pgstat_reset_slru_counter, so why do it
differently here?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-26 02:51 Amit Kapila <[email protected]>
parent: Masahiko Sawada <[email protected]>
2 siblings, 2 replies; 42+ messages in thread
From: Amit Kapila @ 2022-02-26 02:51 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Peter Smith <[email protected]>; [email protected] <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Thu, Feb 24, 2022 at 9:20 PM Masahiko Sawada <[email protected]> wrote:
>
I have reviewed the latest version and made a few changes along with
fixing some of the pending comments by Peter Smith. The changes are as
follows: (a) Removed m_databaseid in PgStat_MsgSubscriptionError as
that is not required now; (b) changed the struct name
PgStat_MsgSubscriptionPurge to PgStat_MsgSubscriptionDrop to make it
similar to DropDb; (c) changed the view name to
pg_stat_subscription_stats, we can reconsider it in future if there is
a consensus on some other name, accordingly changed the reset function
name to pg_stat_reset_subscription_stats; (d) moved some of the newly
added subscription stats functions adjacent to slots to main the
consistency in code; (e) changed comments at few places; (f) added
LATERAL back to system_views query as we refer pg_subscription's oid
in the function call, previously that was not clear.
Do let me know what you think of the attached?
--
With Regards,
Amit Kapila.
Attachments:
[application/octet-stream] v4-0001-Reconsider-pg_stat_subscription_workers-view.patch (73.8K, ../../CAA4eK1K8YfD-girg5upryTDm4dJ6U7pjPR2hX8cbM_g807MDXw@mail.gmail.com/2-v4-0001-Reconsider-pg_stat_subscription_workers-view.patch)
download | inline diff:
From 7172fdf53c6ff91011a6870a94bff1cb8c1e46cf Mon Sep 17 00:00:00 2001
From: Amit Kapila <[email protected]>
Date: Fri, 25 Feb 2022 09:21:11 +0530
Subject: [PATCH v4] Reconsider pg_stat_subscription_workers view.
It was decided (refer to the Discussion link below) that the stats
collector is not an appropriate place to store the error information of
subscription workers.
This patch changes the pg_stat_subscription_workers view (introduced by
commit 8d74fc96db) so that it stores only statistics counters:
apply_error_count and sync_error_count, and has one entry for
each subscription. The removed error information such as error-XID and
the error message would be stored in another way in the future which is
more reliable and persistent.
After removing these error details, there is no longer any relation
information, so the subscription statistics are now a cluster-wide
statistics.
The patch also changes the view name to pg_stat_subscription_stats since
the word "worker" is an implementation detail that we use one worker for
one tablesync and one apply.
Author: Masahiko Sawada, based on suggestions by Andres Freund
Reviewed-by: Peter Smith, Haiying Tang, Amit Kapila
Discussion: https://postgr.es/m/[email protected]
---
doc/src/sgml/logical-replication.sgml | 4 +-
doc/src/sgml/monitoring.sgml | 99 +----
src/backend/catalog/system_functions.sql | 4 +-
src/backend/catalog/system_views.sql | 27 +-
src/backend/postmaster/pgstat.c | 656 +++++++++++++---------------
src/backend/replication/logical/worker.c | 44 +-
src/backend/utils/adt/pgstatfuncs.c | 156 +++----
src/include/catalog/pg_proc.dat | 27 +-
src/include/pgstat.h | 127 ++----
src/test/regress/expected/rules.out | 23 +-
src/test/subscription/t/026_stats.pl | 102 +++++
src/test/subscription/t/026_worker_stats.pl | 165 -------
src/tools/pgindent/typedefs.list | 8 +-
13 files changed, 580 insertions(+), 862 deletions(-)
create mode 100644 src/test/subscription/t/026_stats.pl
delete mode 100644 src/test/subscription/t/026_worker_stats.pl
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 96b4886..fb44723 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -346,9 +346,7 @@
<para>
A conflict will produce an error and will stop the replication; it must be
resolved manually by the user. Details about the conflict can be found in
- <link linkend="monitoring-pg-stat-subscription-workers">
- <structname>pg_stat_subscription_workers</structname></link> and the
- subscriber's server log.
+ the subscriber's server log.
</para>
<para>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index bf7625d..c8897e0 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -628,11 +628,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</row>
<row>
- <entry><structname>pg_stat_subscription_workers</structname><indexterm><primary>pg_stat_subscription_workers</primary></indexterm></entry>
- <entry>One row per subscription worker, showing statistics about errors
- that occurred on that subscription worker.
- See <link linkend="monitoring-pg-stat-subscription-workers">
- <structname>pg_stat_subscription_workers</structname></link> for details.
+ <entry><structname>pg_stat_subscription_stats</structname><indexterm><primary>pg_stat_subscription_stats</primary></indexterm></entry>
+ <entry>One row per subscription, showing statistics about errors.
+ See <link linkend="monitoring-pg-stat-subscription-stats">
+ <structname>pg_stat_subscription_stats</structname></link> for details.
</entry>
</row>
@@ -3063,23 +3062,20 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
</sect2>
- <sect2 id="monitoring-pg-stat-subscription-workers">
- <title><structname>pg_stat_subscription_workers</structname></title>
+ <sect2 id="monitoring-pg-stat-subscription-stats">
+ <title><structname>pg_stat_subscription_stats</structname></title>
<indexterm>
- <primary>pg_stat_subscription_workers</primary>
+ <primary>pg_stat_subscription_stats</primary>
</indexterm>
<para>
- The <structname>pg_stat_subscription_workers</structname> view will contain
- one row per subscription worker on which errors have occurred, for workers
- applying logical replication changes and workers handling the initial data
- copy of the subscribed tables. The statistics entry is removed when the
- corresponding subscription is dropped.
+ The <structname>pg_stat_subscription_stats</structname> view will contain
+ one row per subscription.
</para>
- <table id="pg-stat-subscription-workers" xreflabel="pg_stat_subscription_workers">
- <title><structname>pg_stat_subscription_workers</structname> View</title>
+ <table id="pg-stat-subscription-stats" xreflabel="pg_stat_subscription_stats">
+ <title><structname>pg_stat_subscription_stats</structname> View</title>
<tgroup cols="1">
<thead>
<row>
@@ -3113,72 +3109,31 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>subrelid</structfield> <type>oid</type>
+ <structfield>apply_error_count</structfield> <type>bigint</type>
</para>
<para>
- OID of the relation that the worker is synchronizing; null for the
- main apply worker
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_relid</structfield> <type>oid</type>
- </para>
- <para>
- OID of the relation that the worker was processing when the
- error occurred
+ Number of times an error occurred during the application of changes
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_command</structfield> <type>text</type>
+ <structfield>sync_error_count</structfield> <type>bigint</type>
</para>
<para>
- Name of command being applied when the error occurred. This field
- is null if the error was reported during the initial data copy.
+ Number of times an error occurred during the initial table
+ synchronization
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_xid</structfield> <type>xid</type>
- </para>
- <para>
- Transaction ID of the publisher node being applied when the error
- occurred. This field is null if the error was reported
- during the initial data copy.
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_count</structfield> <type>uint8</type>
- </para>
- <para>
- Number of consecutive times the error occurred
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_message</structfield> <type>text</type>
- </para>
- <para>
- The error message
- </para></entry>
- </row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>last_error_time</structfield> <type>timestamp with time zone</type>
+ <structfield>stats_reset</structfield> <type>timestamp with time zone</type>
</para>
<para>
- Last time at which this error occurred
+ Time at which these statistics were last reset
</para></entry>
</row>
-
</tbody>
</tgroup>
</table>
@@ -5320,22 +5275,16 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
- <primary>pg_stat_reset_subscription_worker</primary>
+ <primary>pg_stat_reset_subscription_stats</primary>
</indexterm>
- <function>pg_stat_reset_subscription_worker</function> ( <parameter>subid</parameter> <type>oid</type> <optional>, <parameter>relid</parameter> <type>oid</type> </optional> )
+ <function>pg_stat_reset_subscription_stats</function> ( <type>oid</type> )
<returnvalue>void</returnvalue>
</para>
<para>
- Resets the statistics of subscription workers running on the
- subscription with <parameter>subid</parameter> shown in the
- <structname>pg_stat_subscription_workers</structname> view. If the
- argument <parameter>relid</parameter> is not <literal>NULL</literal>,
- resets statistics of the subscription worker handling the initial data
- copy of the relation with <parameter>relid</parameter>. Otherwise,
- resets the subscription worker statistics of the main apply worker.
- If the argument <parameter>relid</parameter> is omitted, resets the
- statistics of all subscription workers running on the subscription
- with <parameter>subid</parameter>.
+ Resets statistics for a single subscription shown in the
+ <structname>pg_stat_subscription_stats</structname> view to zero. If
+ the argument is <literal>NULL</literal>, reset statistics for all
+ subscriptions.
</para>
<para>
This function is restricted to superusers by default, but other users
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index fd14217..758ab6e 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -639,9 +639,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_reset_single_function_counters(oid) FROM publ
REVOKE EXECUTE ON FUNCTION pg_stat_reset_replication_slot(text) FROM public;
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid) FROM public;
-
-REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_worker(oid, oid) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_stat_reset_subscription_stats(oid) FROM public;
REVOKE EXECUTE ON FUNCTION lo_import(text) FROM public;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 3cb69b1..d7d5990 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1264,25 +1264,12 @@ GRANT SELECT (oid, subdbid, subname, subowner, subenabled, subbinary,
substream, subtwophasestate, subslotname, subsynccommit, subpublications)
ON pg_subscription TO public;
-CREATE VIEW pg_stat_subscription_workers AS
+CREATE VIEW pg_stat_subscription_stats AS
SELECT
- w.subid,
+ a.subid,
s.subname,
- w.subrelid,
- w.last_error_relid,
- w.last_error_command,
- w.last_error_xid,
- w.last_error_count,
- w.last_error_message,
- w.last_error_time
- FROM (SELECT
- oid as subid,
- NULL as relid
- FROM pg_subscription
- UNION ALL
- SELECT
- srsubid as subid,
- srrelid as relid
- FROM pg_subscription_rel) sr,
- LATERAL pg_stat_get_subscription_worker(sr.subid, sr.relid) w
- JOIN pg_subscription s ON (w.subid = s.oid);
+ a.apply_error_count,
+ a.sync_error_count,
+ a.stats_reset
+ FROM pg_subscription as s,
+ LATERAL pg_stat_get_subscription_stats(s.oid) as a;
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 0646f53..53ddd93 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -106,7 +106,7 @@
#define PGSTAT_DB_HASH_SIZE 16
#define PGSTAT_TAB_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
-#define PGSTAT_SUBWORKER_HASH_SIZE 32
+#define PGSTAT_SUBSCRIPTION_HASH_SIZE 32
#define PGSTAT_REPLSLOT_HASH_SIZE 32
@@ -284,6 +284,7 @@ static PgStat_GlobalStats globalStats;
static PgStat_WalStats walStats;
static PgStat_SLRUStats slruStats[SLRU_NUM_ELEMENTS];
static HTAB *replSlotStatHash = NULL;
+static HTAB *subscriptionStatHash = NULL;
/*
* List of OIDs of databases we need to write out. If an entry is InvalidOid,
@@ -322,14 +323,13 @@ NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_no
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
Oid tableoid, bool create);
-static PgStat_StatSubWorkerEntry *pgstat_get_subworker_entry(PgStat_StatDBEntry *dbentry,
- Oid subid, Oid subrelid,
- bool create);
+static PgStat_StatSubEntry *pgstat_get_subscription_entry(Oid subid, bool create);
+static void pgstat_reset_subscription(PgStat_StatSubEntry *subentry, TimestampTz ts);
static void pgstat_write_statsfiles(bool permanent, bool allDbs);
static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- HTAB *subworkerhash, bool permanent);
+ bool permanent);
static void backend_read_statsfile(void);
static bool pgstat_write_statsfile_needed(void);
@@ -341,7 +341,6 @@ static void pgstat_reset_replslot(PgStat_StatReplSlotEntry *slotstats, Timestamp
static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg, TimestampTz now);
static void pgstat_send_funcstats(void);
static void pgstat_send_slru(void);
-static void pgstat_send_subscription_purge(PgStat_MsgSubscriptionPurge *msg);
static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
static bool pgstat_should_report_connstat(void);
static void pgstat_report_disconnect(Oid dboid);
@@ -363,6 +362,7 @@ static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, in
static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
static void pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len);
static void pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg, int len);
+static void pgstat_recv_resetsubcounter(PgStat_MsgResetsubcounter *msg, int len);
static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
@@ -380,8 +380,8 @@ static void pgstat_recv_connect(PgStat_MsgConnect *msg, int len);
static void pgstat_recv_disconnect(PgStat_MsgDisconnect *msg, int len);
static void pgstat_recv_replslot(PgStat_MsgReplSlot *msg, int len);
static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
-static void pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len);
-static void pgstat_recv_subworker_error(PgStat_MsgSubWorkerError *msg, int len);
+static void pgstat_recv_subscription_drop(PgStat_MsgSubscriptionDrop *msg, int len);
+static void pgstat_recv_subscription_error(PgStat_MsgSubscriptionError *msg, int len);
/* ------------------------------------------------------------
* Public functions called from postmaster follow
@@ -1188,6 +1188,32 @@ pgstat_vacuum_stat(void)
}
/*
+ * Repeat the above steps for subscriptions, if subscription stats are
+ * being collected.
+ */
+ if (subscriptionStatHash)
+ {
+ PgStat_StatSubEntry *subentry;
+
+ /*
+ * Read pg_subscription and make a list of OIDs of all existing
+ * subscriptions.
+ */
+ htab = pgstat_collect_oids(SubscriptionRelationId, Anum_pg_subscription_oid);
+
+ hash_seq_init(&hstat, subscriptionStatHash);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&hstat)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (hash_search(htab, (void *) &(subentry->subid), HASH_FIND, NULL) == NULL)
+ pgstat_report_subscription_drop(subentry->subid);
+ }
+
+ hash_destroy(htab);
+ }
+
+ /*
* Lookup our own database entry; if not found, nothing more to do.
*/
dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
@@ -1311,74 +1337,6 @@ pgstat_vacuum_stat(void)
hash_destroy(htab);
}
-
- /*
- * Repeat for subscription workers. Similarly, we needn't bother in the
- * common case where no subscription workers' stats are being collected.
- */
- if (dbentry->subworkers != NULL &&
- hash_get_num_entries(dbentry->subworkers) > 0)
- {
- PgStat_StatSubWorkerEntry *subwentry;
- PgStat_MsgSubscriptionPurge spmsg;
-
- /*
- * Read pg_subscription and make a list of OIDs of all existing
- * subscriptions
- */
- htab = pgstat_collect_oids(SubscriptionRelationId, Anum_pg_subscription_oid);
-
- spmsg.m_databaseid = MyDatabaseId;
- spmsg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&hstat)) != NULL)
- {
- bool exists = false;
- Oid subid = subwentry->key.subid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &subid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * It is possible that we have multiple entries for the
- * subscription corresponding to apply worker and tablesync
- * workers. In such cases, we don't need to add the same subid
- * again.
- */
- for (int i = 0; i < spmsg.m_nentries; i++)
- {
- if (spmsg.m_subids[i] == subid)
- {
- exists = true;
- break;
- }
- }
-
- if (exists)
- continue;
-
- /* This subscription is dead, add the subid to the message */
- spmsg.m_subids[spmsg.m_nentries++] = subid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (spmsg.m_nentries >= PGSTAT_NUM_SUBSCRIPTIONPURGE)
- {
- pgstat_send_subscription_purge(&spmsg);
- spmsg.m_nentries = 0;
- }
- }
-
- /* Send the rest of dead subscriptions */
- if (spmsg.m_nentries > 0)
- pgstat_send_subscription_purge(&spmsg);
-
- hash_destroy(htab);
- }
}
@@ -1551,8 +1509,7 @@ pgstat_reset_shared_counters(const char *target)
* ----------
*/
void
-pgstat_reset_single_counter(Oid objoid, Oid subobjoid,
- PgStat_Single_Reset_Type type)
+pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
PgStat_MsgResetsinglecounter msg;
@@ -1563,7 +1520,6 @@ pgstat_reset_single_counter(Oid objoid, Oid subobjoid,
msg.m_databaseid = MyDatabaseId;
msg.m_resettype = type;
msg.m_objectid = objoid;
- msg.m_subobjectid = subobjoid;
pgstat_send(&msg, sizeof(msg));
}
@@ -1624,6 +1580,30 @@ pgstat_reset_replslot_counter(const char *name)
}
/* ----------
+ * pgstat_reset_subscription_counter() -
+ *
+ * Tell the statistics collector to reset a single subscription
+ * counter, or all subscription counters (when subid is InvalidOid).
+ *
+ * Permission checking for this function is managed through the normal
+ * GRANT system.
+ * ----------
+ */
+void
+pgstat_reset_subscription_counter(Oid subid)
+{
+ PgStat_MsgResetsubcounter msg;
+
+ if (pgStatSock == PGINVALID_SOCKET)
+ return;
+
+ msg.m_subid = subid;
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSUBCOUNTER);
+
+ pgstat_send(&msg, sizeof(msg));
+}
+
+/* ----------
* pgstat_report_autovac() -
*
* Called from autovacuum.c to report startup of an autovacuum process.
@@ -1949,31 +1929,20 @@ pgstat_report_replslot_drop(const char *slotname)
}
/* ----------
- * pgstat_report_subworker_error() -
+ * pgstat_report_subscription_error() -
*
- * Tell the collector about the subscription worker error.
+ * Tell the collector about the subscription error.
* ----------
*/
void
-pgstat_report_subworker_error(Oid subid, Oid subrelid, Oid relid,
- LogicalRepMsgType command, TransactionId xid,
- const char *errmsg)
+pgstat_report_subscription_error(Oid subid, bool is_apply_error)
{
- PgStat_MsgSubWorkerError msg;
- int len;
+ PgStat_MsgSubscriptionError msg;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBWORKERERROR);
- msg.m_databaseid = MyDatabaseId;
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBSCRIPTIONERROR);
msg.m_subid = subid;
- msg.m_subrelid = subrelid;
- msg.m_relid = relid;
- msg.m_command = command;
- msg.m_xid = xid;
- msg.m_timestamp = GetCurrentTimestamp();
- strlcpy(msg.m_message, errmsg, PGSTAT_SUBWORKERERROR_MSGLEN);
-
- len = offsetof(PgStat_MsgSubWorkerError, m_message) + strlen(msg.m_message) + 1;
- pgstat_send(&msg, len);
+ msg.m_is_apply_error = is_apply_error;
+ pgstat_send(&msg, sizeof(PgStat_MsgSubscriptionError));
}
/* ----------
@@ -1985,12 +1954,11 @@ pgstat_report_subworker_error(Oid subid, Oid subrelid, Oid relid,
void
pgstat_report_subscription_drop(Oid subid)
{
- PgStat_MsgSubscriptionPurge msg;
+ PgStat_MsgSubscriptionDrop msg;
- msg.m_databaseid = MyDatabaseId;
- msg.m_subids[0] = subid;
- msg.m_nentries = 1;
- pgstat_send_subscription_purge(&msg);
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_SUBSCRIPTIONDROP);
+ msg.m_subid = subid;
+ pgstat_send(&msg, sizeof(PgStat_MsgSubscriptionDrop));
}
/* ----------
@@ -3000,36 +2968,6 @@ pgstat_fetch_stat_funcentry(Oid func_id)
/*
* ---------
- * pgstat_fetch_stat_subworker_entry() -
- *
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for subscription worker or NULL.
- * ---------
- */
-PgStat_StatSubWorkerEntry *
-pgstat_fetch_stat_subworker_entry(Oid subid, Oid subrelid)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *wentry = NULL;
-
- /* Load the stats file if needed */
- backend_read_statsfile();
-
- /*
- * Lookup our database, then find the requested subscription worker stats.
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->subworkers != NULL)
- {
- wentry = pgstat_get_subworker_entry(dbentry, subid, subrelid,
- false);
- }
-
- return wentry;
-}
-
-/*
- * ---------
* pgstat_fetch_stat_archiver() -
*
* Support function for the SQL-callable pgstat* functions. Returns
@@ -3141,6 +3079,23 @@ pgstat_fetch_replslot(NameData slotname)
}
/*
+ * ---------
+ * pgstat_fetch_stat_subscription() -
+ *
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the collected statistics for one subscription or NULL.
+ * ---------
+ */
+PgStat_StatSubEntry *
+pgstat_fetch_stat_subscription(Oid subid)
+{
+ /* Load the stats file if needed */
+ backend_read_statsfile();
+
+ return pgstat_get_subscription_entry(subid, false);
+}
+
+/*
* Shut down a single backend's statistics reporting at process exit.
*
* Flush any remaining statistics counts out to the collector.
@@ -3465,24 +3420,6 @@ pgstat_send_slru(void)
}
}
-/* --------
- * pgstat_send_subscription_purge() -
- *
- * Send a subscription purge message to the collector
- * --------
- */
-static void
-pgstat_send_subscription_purge(PgStat_MsgSubscriptionPurge *msg)
-{
- int len;
-
- len = offsetof(PgStat_MsgSubscriptionPurge, m_subids[0])
- + msg->m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg->m_hdr, PGSTAT_MTYPE_SUBSCRIPTIONPURGE);
- pgstat_send(msg, len);
-}
-
/* ----------
* PgstatCollectorMain() -
*
@@ -3668,6 +3605,10 @@ PgstatCollectorMain(int argc, char *argv[])
len);
break;
+ case PGSTAT_MTYPE_RESETSUBCOUNTER:
+ pgstat_recv_resetsubcounter(&msg.msg_resetsubcounter, len);
+ break;
+
case PGSTAT_MTYPE_AUTOVAC_START:
pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
break;
@@ -3738,12 +3679,12 @@ PgstatCollectorMain(int argc, char *argv[])
pgstat_recv_disconnect(&msg.msg_disconnect, len);
break;
- case PGSTAT_MTYPE_SUBSCRIPTIONPURGE:
- pgstat_recv_subscription_purge(&msg.msg_subscriptionpurge, len);
+ case PGSTAT_MTYPE_SUBSCRIPTIONDROP:
+ pgstat_recv_subscription_drop(&msg.msg_subscriptiondrop, len);
break;
- case PGSTAT_MTYPE_SUBWORKERERROR:
- pgstat_recv_subworker_error(&msg.msg_subworkererror, len);
+ case PGSTAT_MTYPE_SUBSCRIPTIONERROR:
+ pgstat_recv_subscription_error(&msg.msg_subscriptionerror, len);
break;
default:
@@ -3791,8 +3732,7 @@ PgstatCollectorMain(int argc, char *argv[])
/*
* Subroutine to clear stats in a database entry
*
- * Tables, functions, and subscription workers hashes are initialized
- * to empty.
+ * Tables and functions hashes are initialized to empty.
*/
static void
reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
@@ -3845,13 +3785,6 @@ reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
PGSTAT_FUNCTION_HASH_SIZE,
&hash_ctl,
HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(PgStat_StatSubWorkerKey);
- hash_ctl.entrysize = sizeof(PgStat_StatSubWorkerEntry);
- dbentry->subworkers = hash_create("Per-database subscription worker",
- PGSTAT_SUBWORKER_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
}
/*
@@ -3876,7 +3809,7 @@ pgstat_get_db_entry(Oid databaseid, bool create)
/*
* If not found, initialize the new one. This creates empty hash tables
- * for tables, functions, and subscription workers, too.
+ * for tables and functions, too.
*/
if (!found)
reset_dbentry_counters(result);
@@ -3935,48 +3868,6 @@ pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
}
/* ----------
- * pgstat_get_subworker_entry
- *
- * Return subscription worker entry with the given subscription OID and
- * relation OID. If subrelid is InvalidOid, it returns an entry of the
- * apply worker otherwise returns an entry of the table sync worker
- * associated with subrelid. If no subscription worker entry exists,
- * initialize it, if the create parameter is true. Else, return NULL.
- * ----------
- */
-static PgStat_StatSubWorkerEntry *
-pgstat_get_subworker_entry(PgStat_StatDBEntry *dbentry, Oid subid, Oid subrelid,
- bool create)
-{
- PgStat_StatSubWorkerEntry *subwentry;
- PgStat_StatSubWorkerKey key;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- key.subid = subid;
- key.subrelid = subrelid;
- subwentry = (PgStat_StatSubWorkerEntry *) hash_search(dbentry->subworkers,
- (void *) &key,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one */
- if (!found)
- {
- subwentry->last_error_relid = InvalidOid;
- subwentry->last_error_command = 0;
- subwentry->last_error_xid = InvalidTransactionId;
- subwentry->last_error_count = 0;
- subwentry->last_error_time = 0;
- subwentry->last_error_message[0] = '\0';
- }
-
- return subwentry;
-}
-
-/* ----------
* pgstat_write_statsfiles() -
* Write the global statistics file, as well as requested DB files.
*
@@ -4059,8 +3950,8 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
{
/*
- * Write out the table, function, and subscription-worker stats for
- * this DB into the appropriate per-DB stat file, if required.
+ * Write out the table and function stats for this DB into the
+ * appropriate per-DB stat file, if required.
*/
if (allDbs || pgstat_db_requested(dbentry->databaseid))
{
@@ -4096,6 +3987,22 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
}
/*
+ * Write subscription stats struct
+ */
+ if (subscriptionStatHash)
+ {
+ PgStat_StatSubEntry *subentry;
+
+ hash_seq_init(&hstat, subscriptionStatHash);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&hstat)) != NULL)
+ {
+ fputc('S', fpout);
+ rc = fwrite(subentry, sizeof(PgStat_StatSubEntry), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+ }
+
+ /*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
* after each individual fputc or fwrite above.
@@ -4174,10 +4081,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
{
HASH_SEQ_STATUS tstat;
HASH_SEQ_STATUS fstat;
- HASH_SEQ_STATUS sstat;
PgStat_StatTabEntry *tabentry;
PgStat_StatFuncEntry *funcentry;
- PgStat_StatSubWorkerEntry *subwentry;
FILE *fpout;
int32 format_id;
Oid dbid = dbentry->databaseid;
@@ -4233,17 +4138,6 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
}
/*
- * Walk through the database's subscription worker stats table.
- */
- hash_seq_init(&sstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&sstat)) != NULL)
- {
- fputc('S', fpout);
- rc = fwrite(subwentry, sizeof(PgStat_StatSubWorkerEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
* after each individual fputc or fwrite above.
@@ -4301,9 +4195,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
* files after reading; the in-memory status is now authoritative, and the
* files would be out of date in case somebody else reads them.
*
- * If a 'deep' read is requested, table/function/subscription-worker stats are
- * read, otherwise the table/function/subscription-worker hash tables remain
- * empty.
+ * If a 'deep' read is requested, table/function stats are read, otherwise
+ * the table/function hash tables remain empty.
* ----------
*/
static HTAB *
@@ -4482,7 +4375,6 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
dbentry->tables = NULL;
dbentry->functions = NULL;
- dbentry->subworkers = NULL;
/*
* In the collector, disregard the timestamp we read from the
@@ -4494,8 +4386,8 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
dbentry->stats_timestamp = 0;
/*
- * Don't create tables/functions/subworkers hashtables for
- * uninteresting databases.
+ * Don't create tables/functions hashtables for uninteresting
+ * databases.
*/
if (onlydb != InvalidOid)
{
@@ -4520,14 +4412,6 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
&hash_ctl,
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
- hash_ctl.keysize = sizeof(PgStat_StatSubWorkerKey);
- hash_ctl.entrysize = sizeof(PgStat_StatSubWorkerEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->subworkers = hash_create("Per-database subscription worker",
- PGSTAT_SUBWORKER_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
/*
* If requested, read the data from the database-specific
* file. Otherwise we just leave the hashtables empty.
@@ -4536,7 +4420,6 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
pgstat_read_db_statsfile(dbentry->databaseid,
dbentry->tables,
dbentry->functions,
- dbentry->subworkers,
permanent);
break;
@@ -4580,6 +4463,45 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
break;
}
+ /*
+ * 'S' A PgStat_StatSubEntry struct describing subscription
+ * statistics.
+ */
+ case 'S':
+ {
+ PgStat_StatSubEntry subbuf;
+ PgStat_StatSubEntry *subentry;
+
+ if (fread(&subbuf, 1, sizeof(PgStat_StatSubEntry), fpin)
+ != sizeof(PgStat_StatSubEntry))
+ {
+ ereport(pgStatRunningInCollector ? LOG : WARNING,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ if (subscriptionStatHash == NULL)
+ {
+ HASHCTL hash_ctl;
+
+ hash_ctl.keysize = sizeof(Oid);
+ hash_ctl.entrysize = sizeof(PgStat_StatSubEntry);
+ hash_ctl.hcxt = pgStatLocalContext;
+ subscriptionStatHash = hash_create("Subscription hash",
+ PGSTAT_SUBSCRIPTION_HASH_SIZE,
+ &hash_ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
+
+ subentry = (PgStat_StatSubEntry *) hash_search(subscriptionStatHash,
+ (void *) &subbuf.subid,
+ HASH_ENTER, NULL);
+
+ memcpy(subentry, &subbuf, sizeof(subbuf));
+ break;
+ }
+
case 'E':
goto done;
@@ -4614,21 +4536,19 @@ done:
* As in pgstat_read_statsfiles, if the permanent file is requested, it is
* removed after reading.
*
- * Note: this code has the ability to skip storing per-table, per-function, or
- * per-subscription-worker data, if NULL is passed for the corresponding hashtable.
- * That's not used at the moment though.
+ * Note: this code has the ability to skip storing per-table or per-function
+ * data, if NULL is passed for the corresponding hashtable. That's not used
+ * at the moment though.
* ----------
*/
static void
pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- HTAB *subworkerhash, bool permanent)
+ bool permanent)
{
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
- PgStat_StatSubWorkerEntry subwbuf;
- PgStat_StatSubWorkerEntry *subwentry;
FILE *fpin;
int32 format_id;
bool found;
@@ -4743,41 +4663,6 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
break;
/*
- * 'S' A PgStat_StatSubWorkerEntry struct describing
- * subscription worker statistics.
- */
- case 'S':
- if (fread(&subwbuf, 1, sizeof(PgStat_StatSubWorkerEntry),
- fpin) != sizeof(PgStat_StatSubWorkerEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if subscription worker data not wanted.
- */
- if (subworkerhash == NULL)
- break;
-
- subwentry = (PgStat_StatSubWorkerEntry *) hash_search(subworkerhash,
- (void *) &subwbuf.key,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(subwentry, &subwbuf, sizeof(subwbuf));
- break;
-
- /*
* 'E' The EOF marker of a complete stats file.
*/
case 'E':
@@ -4829,6 +4714,7 @@ pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
PgStat_WalStats myWalStats;
PgStat_SLRUStats mySLRUStats[SLRU_NUM_ELEMENTS];
PgStat_StatReplSlotEntry myReplSlotStats;
+ PgStat_StatSubEntry mySubStats;
FILE *fpin;
int32 format_id;
const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
@@ -4959,6 +4845,22 @@ pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
}
break;
+ /*
+ * 'S' A PgStat_StatSubEntry struct describing subscription
+ * statistics follows.
+ */
+ case 'S':
+ if (fread(&mySubStats, 1, sizeof(PgStat_StatSubEntry), fpin)
+ != sizeof(PgStat_StatSubEntry))
+ {
+ ereport(pgStatRunningInCollector ? LOG : WARNING,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ FreeFile(fpin);
+ return false;
+ }
+ break;
+
case 'E':
goto done;
@@ -5164,6 +5066,7 @@ pgstat_clear_snapshot(void)
pgStatLocalContext = NULL;
pgStatDBHash = NULL;
replSlotStatHash = NULL;
+ subscriptionStatHash = NULL;
/*
* Historically the backend_status.c facilities lived in this file, and
@@ -5450,8 +5353,6 @@ pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
hash_destroy(dbentry->tables);
if (dbentry->functions != NULL)
hash_destroy(dbentry->functions);
- if (dbentry->subworkers != NULL)
- hash_destroy(dbentry->subworkers);
if (hash_search(pgStatDBHash,
(void *) &dbid,
@@ -5489,16 +5390,13 @@ pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
hash_destroy(dbentry->tables);
if (dbentry->functions != NULL)
hash_destroy(dbentry->functions);
- if (dbentry->subworkers != NULL)
- hash_destroy(dbentry->subworkers);
dbentry->tables = NULL;
dbentry->functions = NULL;
- dbentry->subworkers = NULL;
/*
* Reset database-level stats, too. This creates empty hash tables for
- * tables, functions, and subscription workers.
+ * tables and functions.
*/
reset_dbentry_counters(dbentry);
}
@@ -5567,14 +5465,6 @@ pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
else if (msg->m_resettype == RESET_FUNCTION)
(void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_SUBWORKER)
- {
- PgStat_StatSubWorkerKey key;
-
- key.subid = msg->m_objectid;
- key.subrelid = msg->m_subobjectid;
- (void) hash_search(dbentry->subworkers, (void *) &key, HASH_REMOVE, NULL);
- }
}
/* ----------
@@ -5645,6 +5535,51 @@ pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg,
}
}
+/* ----------
+ * pgstat_recv_resetsubcounter() -
+ *
+ * Reset some subscription statistics of the cluster.
+ * ----------
+ */
+static void
+pgstat_recv_resetsubcounter(PgStat_MsgResetsubcounter *msg, int len)
+{
+ PgStat_StatSubEntry *subentry;
+ TimestampTz ts;
+
+ /* Return if we don't have replication subscription statistics */
+ if (subscriptionStatHash == NULL)
+ return;
+
+ ts = GetCurrentTimestamp();
+ if (!OidIsValid(msg->m_subid))
+ {
+ HASH_SEQ_STATUS sstat;
+
+ /* Clear all subscription counters */
+ hash_seq_init(&sstat, subscriptionStatHash);
+ while ((subentry = (PgStat_StatSubEntry *) hash_seq_search(&sstat)) != NULL)
+ pgstat_reset_subscription(subentry, ts);
+ }
+ else
+ {
+ /* Get the subscription statistics to reset */
+ subentry = pgstat_get_subscription_entry(msg->m_subid, false);
+
+ /*
+ * Nothing to do if the given subscription entry is not found. This
+ * could happen when the subscription with the subid is removed and
+ * the corresponding statistics entry is also removed before receiving
+ * the reset message.
+ */
+ if (!subentry)
+ return;
+
+ /* Reset the stats for the requested subscription */
+ pgstat_reset_subscription(subentry, ts);
+ }
+}
+
/* ----------
* pgstat_recv_autovac() -
@@ -6118,81 +6053,42 @@ pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
}
/* ----------
- * pgstat_recv_subscription_purge() -
+ * pgstat_recv_subscription_drop() -
*
- * Process a SUBSCRIPTIONPURGE message.
+ * Process a SUBSCRIPTIONDROP message.
* ----------
*/
static void
-pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len)
+pgstat_recv_subscription_drop(PgStat_MsgSubscriptionDrop *msg, int len)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *subwentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /* No need to purge if we don't even know the database */
- if (!dbentry || !dbentry->subworkers)
+ /* Return if we don't have replication subscription statistics */
+ if (subscriptionStatHash == NULL)
return;
- /* Remove all subscription worker statistics for the given subscriptions */
- hash_seq_init(&hstat, dbentry->subworkers);
- while ((subwentry = (PgStat_StatSubWorkerEntry *) hash_seq_search(&hstat)) != NULL)
- {
- for (int i = 0; i < msg->m_nentries; i++)
- {
- if (subwentry->key.subid == msg->m_subids[i])
- {
- (void) hash_search(dbentry->subworkers, (void *) &(subwentry->key),
- HASH_REMOVE, NULL);
- break;
- }
- }
- }
+ /* Remove from hashtable if present; we don't care if it's not */
+ (void) hash_search(subscriptionStatHash, (void *) &(msg->m_subid),
+ HASH_REMOVE, NULL);
}
/* ----------
- * pgstat_recv_subworker_error() -
+ * pgstat_recv_subscription_error() -
*
- * Process a SUBWORKERERROR message.
+ * Process a SUBSCRIPTIONERROR message.
* ----------
*/
static void
-pgstat_recv_subworker_error(PgStat_MsgSubWorkerError *msg, int len)
+pgstat_recv_subscription_error(PgStat_MsgSubscriptionError *msg, int len)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatSubWorkerEntry *subwentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
+ PgStat_StatSubEntry *subentry;
- /* Get the subscription worker stats */
- subwentry = pgstat_get_subworker_entry(dbentry, msg->m_subid,
- msg->m_subrelid, true);
- Assert(subwentry);
-
- if (subwentry->last_error_relid == msg->m_relid &&
- subwentry->last_error_command == msg->m_command &&
- subwentry->last_error_xid == msg->m_xid &&
- strcmp(subwentry->last_error_message, msg->m_message) == 0)
- {
- /*
- * The same error occurred again in succession, just update its
- * timestamp and count.
- */
- subwentry->last_error_count++;
- subwentry->last_error_time = msg->m_timestamp;
- return;
- }
+ /* Get the subscription stats */
+ subentry = pgstat_get_subscription_entry(msg->m_subid, true);
+ Assert(subentry);
- /* Otherwise, update the error information */
- subwentry->last_error_relid = msg->m_relid;
- subwentry->last_error_command = msg->m_command;
- subwentry->last_error_xid = msg->m_xid;
- subwentry->last_error_count = 1;
- subwentry->last_error_time = msg->m_timestamp;
- strlcpy(subwentry->last_error_message, msg->m_message,
- PGSTAT_SUBWORKERERROR_MSGLEN);
+ if (msg->m_is_apply_error)
+ subentry->apply_error_count++;
+ else
+ subentry->sync_error_count++;
}
/* ----------
@@ -6313,6 +6209,68 @@ pgstat_reset_replslot(PgStat_StatReplSlotEntry *slotent, TimestampTz ts)
slotent->stat_reset_timestamp = ts;
}
+/* ----------
+ * pgstat_get_subscription_entry
+ *
+ * Return the subscription statistics entry with the given subscription OID.
+ * If no subscription entry exists, initialize it, if the create parameter is
+ * true. Else, return NULL.
+ * ----------
+ */
+static PgStat_StatSubEntry *
+pgstat_get_subscription_entry(Oid subid, bool create)
+{
+ PgStat_StatSubEntry *subentry;
+ bool found;
+ HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
+
+ if (subscriptionStatHash == NULL)
+ {
+ HASHCTL hash_ctl;
+
+ /*
+ * Quick return NULL if the hash table is empty and the caller didn't
+ * request to create the entry.
+ */
+ if (!create)
+ return NULL;
+
+ hash_ctl.keysize = sizeof(Oid);
+ hash_ctl.entrysize = sizeof(PgStat_StatSubEntry);
+ subscriptionStatHash = hash_create("Subscription hash",
+ PGSTAT_SUBSCRIPTION_HASH_SIZE,
+ &hash_ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ subentry = (PgStat_StatSubEntry *) hash_search(subscriptionStatHash,
+ (void *) &subid,
+ action, &found);
+
+ if (!create && !found)
+ return NULL;
+
+ /* If not found, initialize the new one */
+ if (!found)
+ pgstat_reset_subscription(subentry, 0);
+
+ return subentry;
+}
+
+/* ----------
+ * pgstat_reset_subscription
+ *
+ * Reset the given subscription stats.
+ * ----------
+ */
+static void
+pgstat_reset_subscription(PgStat_StatSubEntry *subentry, TimestampTz ts)
+{
+ subentry->apply_error_count = 0;
+ subentry->sync_error_count = 0;
+ subentry->stat_reset_timestamp = ts;
+}
+
/*
* pgstat_slru_index
*
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 5d9acc6..50880f7 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -3377,7 +3377,6 @@ void
ApplyWorkerMain(Datum main_arg)
{
int worker_slot = DatumGetInt32(main_arg);
- MemoryContext cctx = CurrentMemoryContext;
MemoryContext oldctx;
char originname[NAMEDATALEN];
XLogRecPtr origin_startpos;
@@ -3485,20 +3484,15 @@ ApplyWorkerMain(Datum main_arg)
}
PG_CATCH();
{
- MemoryContext ecxt = MemoryContextSwitchTo(cctx);
- ErrorData *errdata = CopyErrorData();
-
/*
- * Report the table sync error. There is no corresponding message
- * type for table synchronization.
+ * Abort the current transaction so that we send the stats message
+ * in an idle state.
*/
- pgstat_report_subworker_error(MyLogicalRepWorker->subid,
- MyLogicalRepWorker->relid,
- MyLogicalRepWorker->relid,
- 0, /* message type */
- InvalidTransactionId,
- errdata->message);
- MemoryContextSwitchTo(ecxt);
+ AbortOutOfAnyTransaction();
+
+ /* Report the worker failed during table synchronization */
+ pgstat_report_subscription_error(MySubscription->oid, false);
+
PG_RE_THROW();
}
PG_END_TRY();
@@ -3625,22 +3619,14 @@ ApplyWorkerMain(Datum main_arg)
}
PG_CATCH();
{
- /* report the apply error */
- if (apply_error_callback_arg.command != 0)
- {
- MemoryContext ecxt = MemoryContextSwitchTo(cctx);
- ErrorData *errdata = CopyErrorData();
-
- pgstat_report_subworker_error(MyLogicalRepWorker->subid,
- MyLogicalRepWorker->relid,
- apply_error_callback_arg.rel != NULL
- ? apply_error_callback_arg.rel->localreloid
- : InvalidOid,
- apply_error_callback_arg.command,
- apply_error_callback_arg.remote_xid,
- errdata->message);
- MemoryContextSwitchTo(ecxt);
- }
+ /*
+ * Abort the current transaction so that we send the stats message in
+ * an idle state.
+ */
+ AbortOutOfAnyTransaction();
+
+ /* Report the worker failed during the application of the change */
+ pgstat_report_subscription_error(MySubscription->oid, !am_tablesync_worker());
PG_RE_THROW();
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 30e8dfa..fd993d0 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2163,7 +2163,7 @@ pg_stat_reset_single_table_counters(PG_FUNCTION_ARGS)
{
Oid taboid = PG_GETARG_OID(0);
- pgstat_reset_single_counter(taboid, InvalidOid, RESET_TABLE);
+ pgstat_reset_single_counter(taboid, RESET_TABLE);
PG_RETURN_VOID();
}
@@ -2173,38 +2173,11 @@ pg_stat_reset_single_function_counters(PG_FUNCTION_ARGS)
{
Oid funcoid = PG_GETARG_OID(0);
- pgstat_reset_single_counter(funcoid, InvalidOid, RESET_FUNCTION);
+ pgstat_reset_single_counter(funcoid, RESET_FUNCTION);
PG_RETURN_VOID();
}
-Datum
-pg_stat_reset_subscription_worker_subrel(PG_FUNCTION_ARGS)
-{
- Oid subid = PG_GETARG_OID(0);
- Oid relid = PG_ARGISNULL(1) ? InvalidOid : PG_GETARG_OID(1);
-
- pgstat_reset_single_counter(subid, relid, RESET_SUBWORKER);
-
- PG_RETURN_VOID();
-}
-
-/* Reset all subscription worker stats associated with the given subscription */
-Datum
-pg_stat_reset_subscription_worker_sub(PG_FUNCTION_ARGS)
-{
- Oid subid = PG_GETARG_OID(0);
-
- /*
- * Use subscription drop message to remove statistics of all subscription
- * workers.
- */
- pgstat_report_subscription_drop(subid);
-
- PG_RETURN_VOID();
-}
-
-
/* Reset SLRU counters (a specific one or all of them). */
Datum
pg_stat_reset_slru(PG_FUNCTION_ARGS)
@@ -2258,6 +2231,32 @@ pg_stat_reset_replication_slot(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+/* Reset subscription stats (a specific one or all of them) */
+Datum
+pg_stat_reset_subscription_stats(PG_FUNCTION_ARGS)
+{
+ Oid subid;
+
+ if (PG_ARGISNULL(0))
+ {
+ /* Clear all subscription stats */
+ subid = InvalidOid;
+ }
+ else
+ {
+ subid = PG_GETARG_OID(0);
+
+ if (!OidIsValid(subid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid subscription OID %u", subid)));
+ }
+
+ pgstat_reset_subscription_counter(subid);
+
+ PG_RETURN_VOID();
+}
+
Datum
pg_stat_get_archiver(PG_FUNCTION_ARGS)
{
@@ -2400,50 +2399,32 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
}
/*
- * Get the subscription worker statistics for the given subscription
- * (and relation).
+ * Get the subscription statistics for the given subscription. If the
+ * subscription statistics is not available, return all-zeros stats.
*/
Datum
-pg_stat_get_subscription_worker(PG_FUNCTION_ARGS)
+pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_SUBSCRIPTION_WORKER_COLS 8
+#define PG_STAT_GET_SUBSCRIPTION_STATS_COLS 4
Oid subid = PG_GETARG_OID(0);
- Oid subrelid;
TupleDesc tupdesc;
- Datum values[PG_STAT_GET_SUBSCRIPTION_WORKER_COLS];
- bool nulls[PG_STAT_GET_SUBSCRIPTION_WORKER_COLS];
- PgStat_StatSubWorkerEntry *wentry;
- int i;
-
- if (PG_ARGISNULL(1))
- subrelid = InvalidOid;
- else
- subrelid = PG_GETARG_OID(1);
+ Datum values[PG_STAT_GET_SUBSCRIPTION_STATS_COLS];
+ bool nulls[PG_STAT_GET_SUBSCRIPTION_STATS_COLS];
+ PgStat_StatSubEntry *subentry;
+ PgStat_StatSubEntry allzero;
- /* Get subscription worker stats */
- wentry = pgstat_fetch_stat_subworker_entry(subid, subrelid);
-
- /* Return NULL if there is no worker statistics */
- if (wentry == NULL)
- PG_RETURN_NULL();
+ /* Get subscription stats */
+ subentry = pgstat_fetch_stat_subscription(subid);
/* Initialise attributes information in the tuple descriptor */
- tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_WORKER_COLS);
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_GET_SUBSCRIPTION_STATS_COLS);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "subid",
OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subrelid",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 3, "last_error_relid",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 4, "last_error_command",
- TEXTOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 5, "last_error_xid",
- XIDOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 6, "last_error_count",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "apply_error_count",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 7, "last_error_message",
- TEXTOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 8, "last_error_time",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "sync_error_count",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
@@ -2451,46 +2432,27 @@ pg_stat_get_subscription_worker(PG_FUNCTION_ARGS)
MemSet(values, 0, sizeof(values));
MemSet(nulls, 0, sizeof(nulls));
- i = 0;
- /* subid */
- values[i++] = ObjectIdGetDatum(subid);
-
- /* subrelid */
- if (OidIsValid(subrelid))
- values[i++] = ObjectIdGetDatum(subrelid);
- else
- nulls[i++] = true;
-
- /* last_error_relid */
- if (OidIsValid(wentry->last_error_relid))
- values[i++] = ObjectIdGetDatum(wentry->last_error_relid);
- else
- nulls[i++] = true;
-
- /* last_error_command */
- if (wentry->last_error_command != 0)
- values[i++] =
- CStringGetTextDatum(logicalrep_message_type(wentry->last_error_command));
- else
- nulls[i++] = true;
+ if (!subentry)
+ {
+ /* If the subscription is not found, initialise its stats */
+ memset(&allzero, 0, sizeof(PgStat_StatSubEntry));
+ subentry = &allzero;
+ }
- /* last_error_xid */
- if (TransactionIdIsValid(wentry->last_error_xid))
- values[i++] = TransactionIdGetDatum(wentry->last_error_xid);
- else
- nulls[i++] = true;
+ /* subid */
+ values[0] = ObjectIdGetDatum(subid);
- /* last_error_count */
- values[i++] = Int64GetDatum(wentry->last_error_count);
+ /* apply_error_count */
+ values[1] = Int64GetDatum(subentry->apply_error_count);
- /* last_error_message */
- values[i++] = CStringGetTextDatum(wentry->last_error_message);
+ /* sync_error_count */
+ values[2] = Int64GetDatum(subentry->sync_error_count);
- /* last_error_time */
- if (wentry->last_error_time != 0)
- values[i++] = TimestampTzGetDatum(wentry->last_error_time);
+ /* stats_reset */
+ if (subentry->stat_reset_timestamp == 0)
+ nulls[3] = true;
else
- nulls[i++] = true;
+ values[3] = TimestampTzGetDatum(subentry->stat_reset_timestamp);
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7de8cfc..bf88858 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5376,14 +5376,14 @@
proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
-{ oid => '8523', descr => 'statistics: information about subscription worker',
- proname => 'pg_stat_get_subscription_worker', prorows => '1', proisstrict => 'f',
- proretset => 't', provolatile => 's', proparallel => 'r',
- prorettype => 'record', proargtypes => 'oid oid',
- proallargtypes => '{oid,oid,oid,oid,oid,text,xid,int8,text,timestamptz}',
- proargmodes => '{i,i,o,o,o,o,o,o,o,o}',
- proargnames => '{subid,subrelid,subid,subrelid,last_error_relid,last_error_command,last_error_xid,last_error_count,last_error_message,last_error_time}',
- prosrc => 'pg_stat_get_subscription_worker' },
+{ oid => '8523', descr => 'statistics: information about subscription stats',
+ proname => 'pg_stat_get_subscription_stats', proisstrict => 'f',
+ provolatile => 's', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o}',
+ proargnames => '{subid,subid,apply_error_count,sync_error_count,stats_reset}',
+ prosrc => 'pg_stat_get_subscription_stats' },
{ oid => '6118', descr => 'statistics: information about subscription',
proname => 'pg_stat_get_subscription', prorows => '10', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
@@ -5772,15 +5772,10 @@
provolatile => 'v', prorettype => 'void', proargtypes => 'text',
prosrc => 'pg_stat_reset_replication_slot' },
{ oid => '8524',
- descr => 'statistics: reset collected statistics for a single subscription worker',
- proname => 'pg_stat_reset_subscription_worker', proisstrict => 'f',
- provolatile => 'v', prorettype => 'void', proargtypes => 'oid oid',
- prosrc => 'pg_stat_reset_subscription_worker_subrel' },
-{ oid => '8525',
- descr => 'statistics: reset all collected statistics for a single subscription',
- proname => 'pg_stat_reset_subscription_worker',
+ descr => 'statistics: reset collected statistics for a single subscription',
+ proname => 'pg_stat_reset_subscription_stats', proisstrict => 'f',
provolatile => 'v', prorettype => 'void', proargtypes => 'oid',
- prosrc => 'pg_stat_reset_subscription_worker_sub' },
+ prosrc => 'pg_stat_reset_subscription_stats' },
{ oid => '3163', descr => 'current trigger depth',
proname => 'pg_trigger_depth', provolatile => 's', proparallel => 'r',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index e10d202..8f77c70 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -67,6 +67,7 @@ typedef enum StatMsgType
PGSTAT_MTYPE_RESETSINGLECOUNTER,
PGSTAT_MTYPE_RESETSLRUCOUNTER,
PGSTAT_MTYPE_RESETREPLSLOTCOUNTER,
+ PGSTAT_MTYPE_RESETSUBCOUNTER,
PGSTAT_MTYPE_AUTOVAC_START,
PGSTAT_MTYPE_VACUUM,
PGSTAT_MTYPE_ANALYZE,
@@ -84,8 +85,8 @@ typedef enum StatMsgType
PGSTAT_MTYPE_REPLSLOT,
PGSTAT_MTYPE_CONNECT,
PGSTAT_MTYPE_DISCONNECT,
- PGSTAT_MTYPE_SUBSCRIPTIONPURGE,
- PGSTAT_MTYPE_SUBWORKERERROR,
+ PGSTAT_MTYPE_SUBSCRIPTIONDROP,
+ PGSTAT_MTYPE_SUBSCRIPTIONERROR,
} StatMsgType;
/* ----------
@@ -148,8 +149,7 @@ typedef enum PgStat_Shared_Reset_Target
typedef enum PgStat_Single_Reset_Type
{
RESET_TABLE,
- RESET_FUNCTION,
- RESET_SUBWORKER
+ RESET_FUNCTION
} PgStat_Single_Reset_Type;
/* ------------------------------------------------------------
@@ -368,7 +368,6 @@ typedef struct PgStat_MsgResetsinglecounter
Oid m_databaseid;
PgStat_Single_Reset_Type m_resettype;
Oid m_objectid;
- Oid m_subobjectid;
} PgStat_MsgResetsinglecounter;
/* ----------
@@ -395,6 +394,19 @@ typedef struct PgStat_MsgResetreplslotcounter
} PgStat_MsgResetreplslotcounter;
/* ----------
+ * PgStat_MsgResetsubcounter Sent by the backend to tell the collector
+ * to reset subscription counter(s)
+ * ----------
+ */
+typedef struct PgStat_MsgResetsubcounter
+{
+ PgStat_MsgHdr m_hdr;
+ Oid m_subid; /* InvalidOid for clearing all subscription
+ * stats */
+} PgStat_MsgResetsubcounter;
+
+
+/* ----------
* PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
* that a database is going to be processed
* ----------
@@ -542,53 +554,28 @@ typedef struct PgStat_MsgReplSlot
} PgStat_MsgReplSlot;
/* ----------
- * PgStat_MsgSubscriptionPurge Sent by the backend and autovacuum to tell the
- * collector about the dead subscriptions.
+ * PgStat_MsgSubscriptionDrop Sent by the backend and autovacuum to tell the
+ * collector about the dead subscription.
* ----------
*/
-#define PGSTAT_NUM_SUBSCRIPTIONPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) / sizeof(Oid))
-
-typedef struct PgStat_MsgSubscriptionPurge
+typedef struct PgStat_MsgSubscriptionDrop
{
PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_subids[PGSTAT_NUM_SUBSCRIPTIONPURGE];
-} PgStat_MsgSubscriptionPurge;
+ Oid m_subid;
+} PgStat_MsgSubscriptionDrop;
/* ----------
- * PgStat_MsgSubWorkerError Sent by the apply worker or the table sync
- * worker to report the error occurred while
- * processing changes.
+ * PgStat_MsgSubscriptionError Sent by the apply worker or the table sync
+ * worker to report an error on the subscription.
* ----------
*/
-#define PGSTAT_SUBWORKERERROR_MSGLEN 256
-typedef struct PgStat_MsgSubWorkerError
+typedef struct PgStat_MsgSubscriptionError
{
PgStat_MsgHdr m_hdr;
- /*
- * m_subid and m_subrelid are used to determine the subscription and the
- * reporter of the error. m_subrelid is InvalidOid if reported by an apply
- * worker otherwise reported by a table sync worker.
- */
- Oid m_databaseid;
Oid m_subid;
- Oid m_subrelid;
-
- /*
- * Oid of the table that the reporter was actually processing. m_relid can
- * be InvalidOid if an error occurred during worker applying a
- * non-data-modification message such as RELATION.
- */
- Oid m_relid;
-
- LogicalRepMsgType m_command;
- TransactionId m_xid;
- TimestampTz m_timestamp;
- char m_message[PGSTAT_SUBWORKERERROR_MSGLEN];
-} PgStat_MsgSubWorkerError;
+ bool m_is_apply_error;
+} PgStat_MsgSubscriptionError;
/* ----------
* PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
@@ -750,6 +737,7 @@ typedef union PgStat_Msg
PgStat_MsgResetsinglecounter msg_resetsinglecounter;
PgStat_MsgResetslrucounter msg_resetslrucounter;
PgStat_MsgResetreplslotcounter msg_resetreplslotcounter;
+ PgStat_MsgResetsubcounter msg_resetsubcounter;
PgStat_MsgAutovacStart msg_autovacuum_start;
PgStat_MsgVacuum msg_vacuum;
PgStat_MsgAnalyze msg_analyze;
@@ -767,8 +755,8 @@ typedef union PgStat_Msg
PgStat_MsgReplSlot msg_replslot;
PgStat_MsgConnect msg_connect;
PgStat_MsgDisconnect msg_disconnect;
- PgStat_MsgSubscriptionPurge msg_subscriptionpurge;
- PgStat_MsgSubWorkerError msg_subworkererror;
+ PgStat_MsgSubscriptionError msg_subscriptionerror;
+ PgStat_MsgSubscriptionDrop msg_subscriptiondrop;
} PgStat_Msg;
@@ -823,16 +811,11 @@ typedef struct PgStat_StatDBEntry
TimestampTz stats_timestamp; /* time of db stats file update */
/*
- * tables, functions, and subscription workers must be last in the struct,
- * because we don't write the pointers out to the stats file.
- *
- * subworkers is the hash table of PgStat_StatSubWorkerEntry which stores
- * statistics of logical replication workers: apply worker and table sync
- * worker.
+ * tables and functions must be last in the struct, because we don't write
+ * the pointers out to the stats file.
*/
HTAB *tables;
HTAB *functions;
- HTAB *subworkers;
} PgStat_StatDBEntry;
@@ -989,38 +972,17 @@ typedef struct PgStat_StatReplSlotEntry
TimestampTz stat_reset_timestamp;
} PgStat_StatReplSlotEntry;
-/* The lookup key for subscription worker hash table */
-typedef struct PgStat_StatSubWorkerKey
-{
- Oid subid;
-
- /*
- * Oid of the table for which tablesync worker will copy the initial data.
- * An InvalidOid will be assigned for apply workers.
- */
- Oid subrelid;
-} PgStat_StatSubWorkerKey;
-
/*
- * Logical replication apply worker and table sync worker statistics kept in the
- * stats collector.
+ * Subscription statistics kept in the stats collector.
*/
-typedef struct PgStat_StatSubWorkerEntry
+typedef struct PgStat_StatSubEntry
{
- PgStat_StatSubWorkerKey key; /* hash key (must be first) */
+ Oid subid; /* hash key (must be first) */
- /*
- * Subscription worker error statistics representing an error that
- * occurred during application of changes or the initial table
- * synchronization.
- */
- Oid last_error_relid;
- LogicalRepMsgType last_error_command;
- TransactionId last_error_xid;
- PgStat_Counter last_error_count;
- TimestampTz last_error_time;
- char last_error_message[PGSTAT_SUBWORKERERROR_MSGLEN];
-} PgStat_StatSubWorkerEntry;
+ PgStat_Counter apply_error_count;
+ PgStat_Counter sync_error_count;
+ TimestampTz stat_reset_timestamp;
+} PgStat_StatSubEntry;
/*
* Working state needed to accumulate per-function-call timing statistics.
@@ -1111,10 +1073,10 @@ extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
extern void pgstat_reset_shared_counters(const char *);
-extern void pgstat_reset_single_counter(Oid objectid, Oid subobjectid,
- PgStat_Single_Reset_Type type);
+extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_reset_slru_counter(const char *);
extern void pgstat_reset_replslot_counter(const char *name);
+extern void pgstat_reset_subscription_counter(Oid subid);
extern void pgstat_report_connect(Oid dboid);
extern void pgstat_report_autovac(Oid dboid);
@@ -1131,9 +1093,7 @@ extern void pgstat_report_checksum_failure(void);
extern void pgstat_report_replslot(const PgStat_StatReplSlotEntry *repSlotStat);
extern void pgstat_report_replslot_create(const char *slotname);
extern void pgstat_report_replslot_drop(const char *slotname);
-extern void pgstat_report_subworker_error(Oid subid, Oid subrelid, Oid relid,
- LogicalRepMsgType command,
- TransactionId xid, const char *errmsg);
+extern void pgstat_report_subscription_error(Oid subid, bool is_apply_error);
extern void pgstat_report_subscription_drop(Oid subid);
extern void pgstat_initialize(void);
@@ -1226,8 +1186,7 @@ extern void pgstat_send_wal(bool force);
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
-extern PgStat_StatSubWorkerEntry *pgstat_fetch_stat_subworker_entry(Oid subid,
- Oid subrelid);
+extern PgStat_StatSubEntry *pgstat_fetch_stat_subscription(Oid subid);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_BgWriterStats *pgstat_fetch_stat_bgwriter(void);
extern PgStat_CheckpointerStats *pgstat_fetch_stat_checkpointer(void);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 1420288..649631d 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2072,24 +2072,13 @@ pg_stat_subscription| SELECT su.oid AS subid,
st.latest_end_time
FROM (pg_subscription su
LEFT JOIN pg_stat_get_subscription(NULL::oid) st(subid, relid, pid, received_lsn, last_msg_send_time, last_msg_receipt_time, latest_end_lsn, latest_end_time) ON ((st.subid = su.oid)));
-pg_stat_subscription_workers| SELECT w.subid,
+pg_stat_subscription_stats| SELECT a.subid,
s.subname,
- w.subrelid,
- w.last_error_relid,
- w.last_error_command,
- w.last_error_xid,
- w.last_error_count,
- w.last_error_message,
- w.last_error_time
- FROM ( SELECT pg_subscription.oid AS subid,
- NULL::oid AS relid
- FROM pg_subscription
- UNION ALL
- SELECT pg_subscription_rel.srsubid AS subid,
- pg_subscription_rel.srrelid AS relid
- FROM pg_subscription_rel) sr,
- (LATERAL pg_stat_get_subscription_worker(sr.subid, sr.relid) w(subid, subrelid, last_error_relid, last_error_command, last_error_xid, last_error_count, last_error_message, last_error_time)
- JOIN pg_subscription s ON ((w.subid = s.oid)));
+ a.apply_error_count,
+ a.sync_error_count,
+ a.stats_reset
+ FROM pg_subscription s,
+ LATERAL pg_stat_get_subscription_stats(s.oid) a(subid, apply_error_count, sync_error_count, stats_reset);
pg_stat_sys_indexes| SELECT pg_stat_all_indexes.relid,
pg_stat_all_indexes.indexrelid,
pg_stat_all_indexes.schemaname,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
new file mode 100644
index 0000000..a42ea31
--- /dev/null
+++ b/src/test/subscription/t/026_stats.pl
@@ -0,0 +1,102 @@
+
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+# Tests for subscription stats.
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Create publisher node.
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+# Create subscriber node.
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+# Initial table setup on both publisher and subscriber. On subscriber we
+# create the same tables but with primary keys. Also, insert some data that
+# will conflict with the data replicated from publisher later.
+$node_publisher->safe_psql(
+ 'postgres',
+ qq[
+BEGIN;
+CREATE TABLE test_tab1 (a int);
+INSERT INTO test_tab1 VALUES (1);
+COMMIT;
+]);
+$node_subscriber->safe_psql(
+ 'postgres',
+ qq[
+BEGIN;
+CREATE TABLE test_tab1 (a int primary key);
+INSERT INTO test_tab1 VALUES (1);
+COMMIT;
+]);
+
+# Setup publication.
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION tap_pub FOR TABLE test_tab1;");
+
+# There shouldn't be any subscription errors before starting logical replication.
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(1) FROM pg_stat_subscription_stats");
+is($result, qq(0), 'check no subscription error');
+
+# Create subscription. The tablesync for test_tab1 on tap_sub will enter into
+# infinite error loop due to violating the unique constraint.
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub;"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Wait for the tablesync error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT sync_error_count > 0
+FROM pg_stat_subscription_stats
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for tablesync error";
+
+# Truncate test_tab1 so that tablesync worker can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
+
+# Wait for initial tablesync for test_tab1 to finish.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT count(1) = 1 FROM pg_subscription_rel
+WHERE srrelid = 'test_tab1'::regclass AND srsubstate in ('r', 's')
+]) or die "Timed out while waiting for subscriber to synchronize data";
+
+# Check test_tab1 on the subscriber has one row.
+$result = $node_subscriber->safe_psql('postgres', "SELECT a FROM test_tab1");
+is($result, qq(1), 'check the table has now row');
+
+# Insert data to test_tab1 on the publisher, raising an error on the subscriber
+# due to violation of the unique constraint on test_tab1.
+$node_publisher->safe_psql('postgres', "INSERT INTO test_tab1 VALUES (1)");
+
+# Wait for the apply error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT apply_error_count > 0
+FROM pg_stat_subscription_stats
+WHERE subname = 'tap_sub'
+]) or die "Timed out while waiting for apply error";
+
+# Truncate test_tab1 so that apply worker can continue.
+$node_subscriber->safe_psql('postgres', "TRUNCATE test_tab1;");
+
+$node_subscriber->stop('fast');
+$node_publisher->stop('fast');
+
+done_testing();
diff --git a/src/test/subscription/t/026_worker_stats.pl b/src/test/subscription/t/026_worker_stats.pl
deleted file mode 100644
index f72e476..0000000
--- a/src/test/subscription/t/026_worker_stats.pl
+++ /dev/null
@@ -1,165 +0,0 @@
-
-# Copyright (c) 2021-2022, PostgreSQL Global Development Group
-
-# Tests for subscription error stats.
-use strict;
-use warnings;
-use PostgreSQL::Test::Cluster;
-use PostgreSQL::Test::Utils;
-use Test::More;
-
-# Test if the error reported on pg_stat_subscription_workers view is expected.
-sub test_subscription_error
-{
- my ($node, $relname, $command, $xid, $by_apply_worker, $errmsg_prefix, $msg)
- = @_;
-
- my $check_sql = qq[
-SELECT count(1) > 0
-FROM pg_stat_subscription_workers
-WHERE last_error_relid = '$relname'::regclass
- AND starts_with(last_error_message, '$errmsg_prefix')];
-
- # subrelid
- $check_sql .= $by_apply_worker
- ? qq[ AND subrelid IS NULL]
- : qq[ AND subrelid = '$relname'::regclass];
-
- # last_error_command
- $check_sql .= $command eq ''
- ? qq[ AND last_error_command IS NULL]
- : qq[ AND last_error_command = '$command'];
-
- # last_error_xid
- $check_sql .= $xid eq ''
- ? qq[ AND last_error_xid IS NULL]
- : qq[ AND last_error_xid = '$xid'::xid];
-
- # Wait for the particular error statistics to be reported.
- $node->poll_query_until('postgres', $check_sql,
-) or die "Timed out while waiting for " . $msg;
-}
-
-# Create publisher node.
-my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
-$node_publisher->init(allows_streaming => 'logical');
-$node_publisher->start;
-
-# Create subscriber node.
-my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
-$node_subscriber->init(allows_streaming => 'logical');
-
-# The subscriber will enter an infinite error loop, so we don't want
-# to overflow the server log with error messages.
-$node_subscriber->append_conf('postgresql.conf',
- qq[
-wal_retrieve_retry_interval = 2s
-]);
-$node_subscriber->start;
-
-# Initial table setup on both publisher and subscriber. On subscriber we
-# create the same tables but with primary keys. Also, insert some data that
-# will conflict with the data replicated from publisher later.
-$node_publisher->safe_psql(
- 'postgres',
- qq[
-BEGIN;
-CREATE TABLE test_tab1 (a int);
-CREATE TABLE test_tab2 (a int);
-INSERT INTO test_tab1 VALUES (1);
-INSERT INTO test_tab2 VALUES (1);
-COMMIT;
-]);
-$node_subscriber->safe_psql(
- 'postgres',
- qq[
-BEGIN;
-CREATE TABLE test_tab1 (a int primary key);
-CREATE TABLE test_tab2 (a int primary key);
-INSERT INTO test_tab2 VALUES (1);
-COMMIT;
-]);
-
-# Setup publications.
-my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
-$node_publisher->safe_psql(
- 'postgres',
- "CREATE PUBLICATION tap_pub FOR TABLE test_tab1, test_tab2;");
-
-# There shouldn't be any subscription errors before starting logical replication.
-my $result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(1) FROM pg_stat_subscription_workers");
-is($result, qq(0), 'check no subscription error');
-
-# Create subscription. The table sync for test_tab2 on tap_sub will enter into
-# infinite error loop due to violating the unique constraint.
-$node_subscriber->safe_psql(
- 'postgres',
- "CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub;");
-
-$node_publisher->wait_for_catchup('tap_sub');
-
-# Wait for initial table sync for test_tab1 to finish.
-$node_subscriber->poll_query_until(
- 'postgres',
- qq[
-SELECT count(1) = 1 FROM pg_subscription_rel
-WHERE srrelid = 'test_tab1'::regclass AND srsubstate in ('r', 's')
-]) or die "Timed out while waiting for subscriber to synchronize data";
-
-# Check the initial data.
-$result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(a) FROM test_tab1");
-is($result, q(1), 'check initial data are copied to subscriber');
-
-# Insert more data to test_tab1, raising an error on the subscriber due to
-# violation of the unique constraint on test_tab1.
-my $xid = $node_publisher->safe_psql(
- 'postgres',
- qq[
-BEGIN;
-INSERT INTO test_tab1 VALUES (1);
-SELECT pg_current_xact_id()::xid;
-COMMIT;
-]);
-test_subscription_error($node_subscriber, 'test_tab1', 'INSERT', $xid,
- 1, # check apply worker error
- qq(duplicate key value violates unique constraint),
- 'error reported by the apply worker');
-
-# Check the table sync worker's error in the view.
-test_subscription_error($node_subscriber, 'test_tab2', '', '',
- 0, # check tablesync worker error
- qq(duplicate key value violates unique constraint),
- 'the error reported by the table sync worker');
-
-# Test for resetting subscription worker statistics.
-# Truncate test_tab1 and test_tab2 so that applying changes and table sync can
-# continue, respectively.
-$node_subscriber->safe_psql(
- 'postgres',
- "TRUNCATE test_tab1, test_tab2;");
-
-# Wait for the data to be replicated.
-$node_subscriber->poll_query_until(
- 'postgres',
- "SELECT count(1) > 0 FROM test_tab1");
-$node_subscriber->poll_query_until(
- 'postgres',
- "SELECT count(1) > 0 FROM test_tab2");
-
-# There shouldn't be any errors in the view after dropping the subscription.
-$node_subscriber->safe_psql(
- 'postgres',
- "DROP SUBSCRIPTION tap_sub;");
-$result = $node_subscriber->safe_psql(
- 'postgres',
- "SELECT count(1) FROM pg_stat_subscription_workers");
-is($result, q(0), 'no error after dropping subscription');
-
-$node_subscriber->stop('fast');
-$node_publisher->stop('fast');
-
-done_testing();
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c6b302c..d9b83f7 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1943,9 +1943,10 @@ PgStat_MsgResetreplslotcounter
PgStat_MsgResetsharedcounter
PgStat_MsgResetsinglecounter
PgStat_MsgResetslrucounter
+PgStat_MsgResetsubcounter
PgStat_MsgSLRU
-PgStat_MsgSubscriptionPurge
-PgStat_MsgSubWorkerError
+PgStat_MsgSubscriptionDrop
+PgStat_MsgSubscriptionError
PgStat_MsgTabpurge
PgStat_MsgTabstat
PgStat_MsgTempFile
@@ -1957,8 +1958,7 @@ PgStat_Single_Reset_Type
PgStat_StatDBEntry
PgStat_StatFuncEntry
PgStat_StatReplSlotEntry
-PgStat_StatSubWorkerEntry
-PgStat_StatSubWorkerKey
+PgStat_StatSubEntry
PgStat_StatTabEntry
PgStat_SubXactStatus
PgStat_TableCounts
--
1.8.3.1
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-26 02:54 Amit Kapila <[email protected]>
parent: Peter Smith <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Amit Kapila @ 2022-02-26 02:54 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; [email protected] <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Fri, Feb 25, 2022 at 7:26 AM Peter Smith <[email protected]> wrote:
>
> Below are my review comments for the v3 patch.
>
...
> 2. doc/src/sgml/monitoring.sgml
>
> + <entry><structname>pg_stat_subscription_activity</structname><indexterm><primary>pg_stat_subscription_activity</primary></indexterm></entry>
> + <entry>One row per subscription, showing statistics about subscription
> + activity.
> + See <link linkend="monitoring-pg-stat-subscription-activity">
> + <structname>pg_stat_subscription_activity</structname></link>
> for details.
> </entry>
> </row>
>
> Currently these stats are only about errors. These are not really
> statistics about "activity" though. Probably it is better just to
> avoid that word altogether?
>
> SUGGESTIONS
>
> e.g.1. "One row per subscription, showing statistics about
> subscription activity." --> "One row per subscription, showing
> statistics about errors."
>
I preferred this one and made another change suggested by you in the
latest version posted by me. Thanks!
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-28 01:18 Masahiko Sawada <[email protected]>
parent: Amit Kapila <[email protected]>
1 sibling, 0 replies; 42+ messages in thread
From: Masahiko Sawada @ 2022-02-28 01:18 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Peter Smith <[email protected]>; [email protected] <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Sat, Feb 26, 2022 at 11:51 AM Amit Kapila <[email protected]> wrote:
>
> On Thu, Feb 24, 2022 at 9:20 PM Masahiko Sawada <[email protected]> wrote:
> >
>
> I have reviewed the latest version and made a few changes along with
> fixing some of the pending comments by Peter Smith.
Thank you for updating the patch!
> The changes are as
> follows: (a) Removed m_databaseid in PgStat_MsgSubscriptionError as
> that is not required now; (b) changed the struct name
> PgStat_MsgSubscriptionPurge to PgStat_MsgSubscriptionDrop to make it
> similar to DropDb; (c) changed the view name to
> pg_stat_subscription_stats, we can reconsider it in future if there is
> a consensus on some other name, accordingly changed the reset function
> name to pg_stat_reset_subscription_stats; (d) moved some of the newly
> added subscription stats functions adjacent to slots to main the
> consistency in code; (e) changed comments at few places;
Agreed.
> (f) added
> LATERAL back to system_views query as we refer pg_subscription's oid
> in the function call, previously that was not clear.
I think LATERAL is still unnecessary as you pointed out before. The
documentation[1] says,
LATERAL can also precede a function-call FROM item, but in this case
it is a noise word, because the function expression can refer to
earlier FROM items in any case.
The rest looks good to me.
Regards,
[1] https://www.postgresql.org/docs/devel/sql-select.html
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-28 03:29 Peter Smith <[email protected]>
parent: Amit Kapila <[email protected]>
1 sibling, 1 reply; 42+ messages in thread
From: Peter Smith @ 2022-02-28 03:29 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; [email protected] <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
Below are my comments for the v4 patch.
These are only nitpicking comments now; Otherwise, it LGTM.
(Sorry, now I see there are some overlaps with comments posted in the
last 20 mins so take or leave these as you wish)
======
1. doc/src/sgml/monitoring.sgml
- <para>
- OID of the relation that the worker was processing when the
- error occurred
+ Number of times an error occurred during the application of changes
</para></entry>
</row>
BEFORE
Number of times an error occurred during the application of changes
SUGGESTED
Number of times an error occurred while applying changes
~~~
2. doc/src/sgml/monitoring.sgml
+ Resets statistics for a single subscription shown in the
+ <structname>pg_stat_subscription_stats</structname> view to zero. If
+ the argument is <literal>NULL</literal>, reset statistics for all
+ subscriptions.
</para>
SUGGESTED (simpler description, more similar to pg_stat_reset_replication_slot)
Reset statistics to zero for a single subscription. If the argument is
<literal>NULL</literal>, reset statistics for all subscriptions.
~~~
3. src/backend/replication/logical/worker.c - comment
+ /* */
+ pgstat_report_subscription_error(MySubscription->oid, !am_tablesync_worker());
BEFORE
Report the worker failed during the application of the change
SUGGESTED
Report the worker failed while applying changes
~~~
4. src/include/pgstat.h - comment
+typedef struct PgStat_MsgResetsubcounter
+{
+ PgStat_MsgHdr m_hdr;
+ Oid m_subid; /* InvalidOid for clearing all subscription
+ * stats */
+} PgStat_MsgResetsubcounter;
BEFORE
InvalidOid for clearing all subscription stats
SUGGESTED
InvalidOid means reset all subscription stats
------
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 42+ messages in thread
* Re: Design of pg_stat_subscription_workers vs pgstats
@ 2022-02-28 03:50 Amit Kapila <[email protected]>
parent: Peter Smith <[email protected]>
0 siblings, 0 replies; 42+ messages in thread
From: Amit Kapila @ 2022-02-28 03:50 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; [email protected] <[email protected]>; Andres Freund <[email protected]>; David G. Johnston <[email protected]>; pgsql-hackers
On Mon, Feb 28, 2022 at 8:59 AM Peter Smith <[email protected]> wrote:
>
>
> 2. doc/src/sgml/monitoring.sgml
>
> + Resets statistics for a single subscription shown in the
> + <structname>pg_stat_subscription_stats</structname> view to zero. If
> + the argument is <literal>NULL</literal>, reset statistics for all
> + subscriptions.
> </para>
>
> SUGGESTED (simpler description, more similar to pg_stat_reset_replication_slot)
> Reset statistics to zero for a single subscription. If the argument is
> <literal>NULL</literal>, reset statistics for all subscriptions.
>
As discussed, it is better to keep the view name in this description
important as we have another view (pg_stat_susbcription) as well. So,
I am planning to retain the current wording.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 42+ messages in thread
end of thread, other threads:[~2022-02-28 03:50 UTC | newest]
Thread overview: 42+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-02-17 03:07 [PATCH 02/19] doc: Remove 'synchronized' from --no-sync Justin Pryzby <[email protected]>
2022-02-17 03:07 [PATCH 05/17] doc: Remove 'synchronized' from --no-sync Justin Pryzby <[email protected]>
2022-02-17 03:07 [PATCH 01/13] doc: Remove 'synchronized' from --no-sync Justin Pryzby <[email protected]>
2022-02-21 05:34 Re: Design of pg_stat_subscription_workers vs pgstats Andres Freund <[email protected]>
2022-02-21 07:09 ` Re: Design of pg_stat_subscription_workers vs pgstats Amit Kapila <[email protected]>
2022-02-21 07:48 ` Re: Design of pg_stat_subscription_workers vs pgstats Andres Freund <[email protected]>
2022-02-21 09:19 ` Re: Design of pg_stat_subscription_workers vs pgstats Amit Kapila <[email protected]>
2022-02-21 16:07 ` Re: Design of pg_stat_subscription_workers vs pgstats David G. Johnston <[email protected]>
2022-02-22 03:37 ` Re: Design of pg_stat_subscription_workers vs pgstats Amit Kapila <[email protected]>
2022-02-21 16:17 ` Re: Design of pg_stat_subscription_workers vs pgstats Andres Freund <[email protected]>
2022-02-24 09:20 ` Re: Design of pg_stat_subscription_workers vs pgstats Peter Eisentraut <[email protected]>
2022-02-22 05:45 ` Re: Design of pg_stat_subscription_workers vs pgstats Masahiko Sawada <[email protected]>
2022-02-22 09:53 ` Re: Design of pg_stat_subscription_workers vs pgstats Amit Kapila <[email protected]>
2022-02-22 14:32 ` Re: Design of pg_stat_subscription_workers vs pgstats Masahiko Sawada <[email protected]>
2022-02-23 03:47 ` Re: Design of pg_stat_subscription_workers vs pgstats Amit Kapila <[email protected]>
2022-02-22 12:22 ` RE: Design of pg_stat_subscription_workers vs pgstats [email protected] <[email protected]>
2022-02-22 14:46 ` Re: Design of pg_stat_subscription_workers vs pgstats Masahiko Sawada <[email protected]>
2022-02-23 01:13 ` RE: Design of pg_stat_subscription_workers vs pgstats [email protected] <[email protected]>
2022-02-23 02:14 ` Re: Design of pg_stat_subscription_workers vs pgstats Andres Freund <[email protected]>
2022-02-23 02:46 ` Re: Design of pg_stat_subscription_workers vs pgstats Masahiko Sawada <[email protected]>
2022-02-24 09:48 ` Re: Design of pg_stat_subscription_workers vs pgstats Peter Eisentraut <[email protected]>
2022-02-23 03:00 ` RE: Design of pg_stat_subscription_workers vs pgstats [email protected] <[email protected]>
2022-02-23 03:07 ` Re: Design of pg_stat_subscription_workers vs pgstats Peter Smith <[email protected]>
2022-02-23 06:20 ` Re: Design of pg_stat_subscription_workers vs pgstats Peter Smith <[email protected]>
2022-02-24 01:32 ` Re: Design of pg_stat_subscription_workers vs pgstats Masahiko Sawada <[email protected]>
2022-02-24 07:20 ` RE: Design of pg_stat_subscription_workers vs pgstats [email protected] <[email protected]>
2022-02-24 08:53 ` Re: Design of pg_stat_subscription_workers vs pgstats Peter Smith <[email protected]>
2022-02-24 12:05 ` Re: Design of pg_stat_subscription_workers vs pgstats Amit Kapila <[email protected]>
2022-02-24 12:17 ` Re: Design of pg_stat_subscription_workers vs pgstats Masahiko Sawada <[email protected]>
2022-02-24 15:49 ` Re: Design of pg_stat_subscription_workers vs pgstats Masahiko Sawada <[email protected]>
2022-02-25 01:56 ` Re: Design of pg_stat_subscription_workers vs pgstats Peter Smith <[email protected]>
2022-02-26 02:54 ` Re: Design of pg_stat_subscription_workers vs pgstats Amit Kapila <[email protected]>
2022-02-25 05:26 ` Re: Design of pg_stat_subscription_workers vs pgstats Amit Kapila <[email protected]>
2022-02-26 02:51 ` Re: Design of pg_stat_subscription_workers vs pgstats Amit Kapila <[email protected]>
2022-02-28 01:18 ` Re: Design of pg_stat_subscription_workers vs pgstats Masahiko Sawada <[email protected]>
2022-02-28 03:29 ` Re: Design of pg_stat_subscription_workers vs pgstats Peter Smith <[email protected]>
2022-02-28 03:50 ` Re: Design of pg_stat_subscription_workers vs pgstats Amit Kapila <[email protected]>
2022-02-24 09:53 ` Re: Design of pg_stat_subscription_workers vs pgstats Peter Eisentraut <[email protected]>
2022-02-24 11:46 ` Re: Design of pg_stat_subscription_workers vs pgstats Masahiko Sawada <[email protected]>
2022-02-24 12:23 ` Re: Design of pg_stat_subscription_workers vs pgstats Peter Eisentraut <[email protected]>
2022-02-24 12:51 ` Re: Design of pg_stat_subscription_workers vs pgstats Masahiko Sawada <[email protected]>
2022-02-24 14:45 ` Re: Design of pg_stat_subscription_workers vs pgstats Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox