public inbox for [email protected]help / color / mirror / Atom feed
Fix data checksum progress reporting docs 6+ messages / 2 participants [nested] [flat]
* Fix data checksum progress reporting docs @ 2026-07-08 12:23 Fujii Masao <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Fujii Masao @ 2026-07-08 12:23 UTC (permalink / raw) To: [email protected] <[email protected]> Hi, While trying the new features in v19, I found a few documentation issues with pg_stat_progress_data_checksums: - pg_stat_progress_data_checksums has its own section, but is missing from the progress view summary table. - The general progress reporting overview does not mention online data checksum operations. - The section describes only checksum enabling, even though the view also reports checksum disabling. - The progress counter columns are documented as integer, but are actually bigint. The attached patch updates monitoring.sgml to address these issues. Regards, -- Fujii Masao Attachments: [application/octet-stream] v1-0001-doc-Fix-data-checksum-progress-reporting-document.patch (5.7K, ../../CAHGQGwHJHJYAkYZBi3_O13np-Rou9UL637=hB3Y_-qdCgcZn-w@mail.gmail.com/2-v1-0001-doc-Fix-data-checksum-progress-reporting-document.patch) download | inline diff: From 2bd5f1762dd9a940ec28f614517bc38a86e6a88d Mon Sep 17 00:00:00 2001 From: Fujii Masao <[email protected]> Date: Wed, 8 Jul 2026 20:29:53 +0900 Subject: [PATCH v1] doc: Fix data checksum progress reporting documentation Add pg_stat_progress_data_checksums to the progress reporting summary and the list of commands with progress reporting. Also clarify that the view reports both enabling and disabling data checksums, and correct the documented types of its progress counters to bigint. --- doc/src/sgml/monitoring.sgml | 39 +++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 12b9ee20d4a..858788b227c 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -435,6 +435,16 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser See <xref linkend="copy-progress-reporting"/>. </entry> </row> + + <row> + <entry><structname>pg_stat_progress_data_checksums</structname><indexterm><primary>pg_stat_progress_data_checksums</primary></indexterm></entry> + <entry>One row for the data checksum launcher process while data + checksums are being enabled or disabled. When enabling data + checksums, the view also has one row for each worker process, + showing current progress. + See <xref linkend="data-checksum-progress-reporting"/>. + </entry> + </row> </tbody> </tgroup> </table> @@ -6271,9 +6281,9 @@ FROM pg_stat_get_backend_idset() AS backendid; <command>COPY</command>, <command>CREATE INDEX</command>, <command>REPACK</command> (and its obsolete spelling <command>CLUSTER</command>), <command>VACUUM</command>, - and <xref linkend="protocol-replication-base-backup"/> (i.e., replication + <xref linkend="protocol-replication-base-backup"/> (i.e., replication command that <xref linkend="app-pgbasebackup"/> issues to take - a base backup). + a base backup), and online data checksum operations. This may be expanded in the future. </para> @@ -8005,13 +8015,14 @@ FROM pg_stat_get_backend_idset() AS backendid; </indexterm> <para> - When data checksums are being enabled on a running cluster, the + When data checksums are being enabled or disabled on a running cluster, the <structname>pg_stat_progress_data_checksums</structname> view will contain - a row for the launcher process, and one row for each worker process which - is currently calculating and writing checksums for the data pages in a database. - The launcher provides overview of the overall progress (how many databases - have been processed, how many remain), while the workers track progress for - currently processed databases. + a row for the launcher process. When enabling data checksums, the view + will also contain one row for each worker process which is currently + calculating and writing checksums for the data pages in a database. The + launcher provides an overview of the overall progress, such as how many + databases have been processed and how many remain, while the workers track + progress for currently processed databases. </para> <table id="pg-stat-progress-data-checksums-view" xreflabel="pg_stat_progress_data_checksums"> @@ -8080,7 +8091,7 @@ FROM pg_stat_get_backend_idset() AS backendid; <row> <entry role="catalog_table_entry"> <para role="column_definition"> - <structfield>databases_total</structfield> <type>integer</type> + <structfield>databases_total</structfield> <type>bigint</type> </para> <para> The total number of databases which will be processed. Only the @@ -8093,7 +8104,7 @@ FROM pg_stat_get_backend_idset() AS backendid; <row> <entry role="catalog_table_entry"> <para role="column_definition"> - <structfield>databases_done</structfield> <type>integer</type> + <structfield>databases_done</structfield> <type>bigint</type> </para> <para> The number of databases which have been processed. Only the launcher @@ -8106,7 +8117,7 @@ FROM pg_stat_get_backend_idset() AS backendid; <row> <entry role="catalog_table_entry"> <para role="column_definition"> - <structfield>relations_total</structfield> <type>integer</type> + <structfield>relations_total</structfield> <type>bigint</type> </para> <para> The total number of relations which will be processed, or @@ -8121,7 +8132,7 @@ FROM pg_stat_get_backend_idset() AS backendid; <row> <entry role="catalog_table_entry"> <para role="column_definition"> - <structfield>relations_done</structfield> <type>integer</type> + <structfield>relations_done</structfield> <type>bigint</type> </para> <para> The number of relations which have been processed. The launcher @@ -8133,7 +8144,7 @@ FROM pg_stat_get_backend_idset() AS backendid; <row> <entry role="catalog_table_entry"> <para role="column_definition"> - <structfield>blocks_total</structfield> <type>integer</type> + <structfield>blocks_total</structfield> <type>bigint</type> </para> <para> The number of blocks in the current relation which will be processed, @@ -8147,7 +8158,7 @@ FROM pg_stat_get_backend_idset() AS backendid; <row> <entry role="catalog_table_entry"> <para role="column_definition"> - <structfield>blocks_done</structfield> <type>integer</type> + <structfield>blocks_done</structfield> <type>bigint</type> </para> <para> The number of blocks in the current relation which have been processed. -- 2.55.0 ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Fix data checksum progress reporting docs @ 2026-07-08 13:24 Daniel Gustafsson <[email protected]> parent: Fujii Masao <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Daniel Gustafsson @ 2026-07-08 13:24 UTC (permalink / raw) To: Fujii Masao <[email protected]>; +Cc: [email protected] <[email protected]> > On 8 Jul 2026, at 14:23, Fujii Masao <[email protected]> wrote: > While trying the new features in v19, I found a few documentation > issues with pg_stat_progress_data_checksums: Thanks a lot for post-commit review! > - pg_stat_progress_data_checksums has its own section, but is missing > from the progress view summary table. > > - The general progress reporting overview does not mention online data > checksum operations. > > - The section describes only checksum enabling, even though the view > also reports checksum disabling. > > - The progress counter columns are documented as integer, but are > actually bigint. The proposed changes all look good to me, thanks! -- Daniel Gustafsson ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Fix data checksum progress reporting docs @ 2026-07-09 00:17 Fujii Masao <[email protected]> parent: Daniel Gustafsson <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Fujii Masao @ 2026-07-09 00:17 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; +Cc: [email protected] <[email protected]> On Wed, Jul 8, 2026 at 10:24 PM Daniel Gustafsson <[email protected]> wrote: > The proposed changes all look good to me, thanks! Thanks for the review! I've pushed the patch. BTW, this is a separate issue, but also related to the data checksums docs, I found that the glossary entry for "Auxiliary process" includes the data checksums worker and data checksums worker launcher. However, their own glossary entries describe them as background workers, and they are implemented as dynamic background workers. This seems inconsistent. Other dynamic background workers, such as logical replication apply workers, are not listed as auxiliary processes. I don't see why the data checksums worker and launcher should be treated differently. Wouldn't it be better to remove the data checksums worker and data checksums worker launcher from the "Auxiliary process" entry? Regards, -- Fujii Masao ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Fix data checksum progress reporting docs @ 2026-07-09 08:28 Daniel Gustafsson <[email protected]> parent: Fujii Masao <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Daniel Gustafsson @ 2026-07-09 08:28 UTC (permalink / raw) To: Fujii Masao <[email protected]>; +Cc: [email protected] <[email protected]> > On 9 Jul 2026, at 02:17, Fujii Masao <[email protected]> wrote: > > On Wed, Jul 8, 2026 at 10:24 PM Daniel Gustafsson <[email protected]> wrote: >> The proposed changes all look good to me, thanks! > > Thanks for the review! I've pushed the patch. > > BTW, this is a separate issue, but also related to the data checksums docs, > I found that the glossary entry for "Auxiliary process" includes > the data checksums worker and data checksums worker launcher. > However, their own glossary entries describe them as background workers, > and they are implemented as dynamic background workers. > > This seems inconsistent. Other dynamic background workers, such as > logical replication apply workers, are not listed as auxiliary processes. > I don't see why the data checksums worker and launcher should be treated > differently. > > Wouldn't it be better to remove the data checksums worker and data > checksums worker launcher from the "Auxiliary process" entry? You are absolutely right. And when looking at it I think we can simplify the entry a bit as well to make it fit the style of the page better. What do you think about the attached? -- Daniel Gustafsson Attachments: [application/octet-stream] 0001-doc-Fix-glossary-entry-for-data-checksums-workers.patch (3.7K, ../../[email protected]/2-0001-doc-Fix-glossary-entry-for-data-checksums-workers.patch) download | inline diff: From 740c9c8e44dc7c419a2ceb0207c43f7588229e87 Mon Sep 17 00:00:00 2001 From: Daniel Gustafsson <[email protected]> Date: Thu, 9 Jul 2026 10:23:22 +0200 Subject: [PATCH] doc: Fix glossary entry for data checksums workers The glossary entry for data checksums workers incorrectly stated that they were auxiliary processes, but they are implementerd as background workers. Fix, and while there, simplify the entry by combining the worker and launcher into a single glossary term. Reported-by: Fujii Masao <[email protected]> Discussion: https://postgr.es/m/CAHGQGwEv-C9ia+rBYyePzO8F=5FVvS412ZqcOupazuOb5RafNg@mail.gmail.com --- doc/src/sgml/glossary.sgml | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml index f7f69afc922..4f841fb5886 100644 --- a/doc/src/sgml/glossary.sgml +++ b/doc/src/sgml/glossary.sgml @@ -199,8 +199,6 @@ (but not the autovacuum workers), the <glossterm linkend="glossary-background-writer">background writer</glossterm>, the <glossterm linkend="glossary-checkpointer">checkpointer</glossterm>, - the <glossterm linkend="glossary-data-checksums-worker">data checksums worker</glossterm>, - the <glossterm linkend="glossary-data-checksums-worker-launcher">data checksums worker launcher</glossterm>, the <glossterm linkend="glossary-logger">logger</glossterm>, the <glossterm linkend="glossary-startup-process">startup process</glossterm>, the <glossterm linkend="glossary-wal-archiver">WAL archiver</glossterm>, @@ -236,8 +234,9 @@ which runs system- or user-supplied code. Serves as infrastructure for several features in <productname>PostgreSQL</productname>, such as - <glossterm linkend="glossary-replication">logical replication</glossterm> - and <glossterm linkend="glossary-parallel-query">parallel queries</glossterm>. + <glossterm linkend="glossary-replication">logical replication</glossterm>, + <glossterm linkend="glossary-parallel-query">parallel queries</glossterm> and + <glossterm linkend="glossary-data-checksums-process">data checksums processing</glossterm>. In addition, <glossterm linkend="glossary-extension">Extensions</glossterm> can add custom background worker processes. </para> @@ -588,24 +587,17 @@ <glosssee otherterm="glossary-data-directory" /> </glossentry> - <glossentry id="glossary-data-checksums-worker"> - <glossterm>Data Checksums Worker</glossterm> + <glossentry id="glossary-data-checksums-process"> + <glossterm>Data Checksums (process)</glossterm> <glossdef> <para> - A <glossterm linkend="glossary-background-worker">background worker</glossterm> - which enables data checksums in a specific database. - </para> - </glossdef> - </glossentry> - - <glossentry id="glossary-data-checksums-worker-launcher"> - <glossterm>Data Checksums Worker Launcher</glossterm> - <glossdef> - <para> - A <glossterm linkend="glossary-background-worker">background worker</glossterm> - which starts <glossterm linkend="glossary-data-checksums-worker"> data - checksum worker</glossterm> processes for enabling data checksums in each - database, or disables data checksums cluster-wide. + A set of + <glossterm linkend="glossary-background-worker">background worker</glossterm> + processes which can enable, or disable, data checksums in a + running cluster. The process which coordinates the work is known as the + <firstterm>data checksums worker launcher</firstterm> and the process + which operate on the individual databases is known as the + <firstterm>data checksums worker</firstterm>. </para> </glossdef> </glossentry> -- 2.39.3 (Apple Git-146) ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Fix data checksum progress reporting docs @ 2026-07-10 01:13 Fujii Masao <[email protected]> parent: Daniel Gustafsson <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Fujii Masao @ 2026-07-10 01:13 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; +Cc: [email protected] <[email protected]> On Thu, Jul 9, 2026 at 5:28 PM Daniel Gustafsson <[email protected]> wrote: > You are absolutely right. And when looking at it I think we can simplify the > entry a bit as well to make it fit the style of the page better. What do you > think about the attached? Thanks for the patch! I like the overall approach. I just have a few minor comments. + A set of + <glossterm linkend="glossary-background-worker">background worker</glossterm> + processes which can enable, or disable, data checksums in a Wouldn't it be better to remove the two commas here? + running cluster. The process which coordinates the work is known as the + <firstterm>data checksums worker launcher</firstterm> and the process Only the glossary seems to use the term "data checksums worker launcher". Wouldn't "data checksums launcher" (i.e., dropping "worker") be a better name? + <firstterm>data checksums worker launcher</firstterm> and the process + which operate on the individual databases is known as the + <firstterm>data checksums worker</firstterm>. Shouldn't "the process which operate" be either "the process which operates" or "the processes which operate"? Since multiple data checksums workers can run, the latter seems more appropriate. While looking at the related code, I also found that B_DATACHECKSUMSWORKER_LAUNCHER and B_DATACHECKSUMSWORKER_WORKER were added as BackendType entries in miscadmin.h. Since those entries appear under the comment for auxiliary processes, it could give the impression that the data checksums launcher and workers are auxiliary processes. To avoid that confusion, would it make sense to add a comment such as: ------------------------- B_WAL_WRITER, + /* + * XXXXXXXXXX + */ B_DATACHECKSUMSWORKER_LAUNCHER, B_DATACHECKSUMSWORKER_WORKER, ------------------------- BTW, I'm also wondering why dedicated BackendType entries were added for the data checksums processes, while other background workers don't have their own BackendType values. Regards, -- Fujii Masao ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Fix data checksum progress reporting docs @ 2026-07-10 10:00 Daniel Gustafsson <[email protected]> parent: Fujii Masao <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Daniel Gustafsson @ 2026-07-10 10:00 UTC (permalink / raw) To: Fujii Masao <[email protected]>; +Cc: [email protected] <[email protected]> > On 10 Jul 2026, at 03:13, Fujii Masao <[email protected]> wrote: > > On Thu, Jul 9, 2026 at 5:28 PM Daniel Gustafsson <[email protected]> wrote: >> You are absolutely right. And when looking at it I think we can simplify the >> entry a bit as well to make it fit the style of the page better. What do you >> think about the attached? > > Thanks for the patch! I like the overall approach. > I just have a few minor comments. > > + A set of > + <glossterm linkend="glossary-background-worker">background > worker</glossterm> > + processes which can enable, or disable, data checksums in a > > Wouldn't it be better to remove the two commas here? Fixed. > + running cluster. The process which coordinates the work is known as the > + <firstterm>data checksums worker launcher</firstterm> and the process > > Only the glossary seems to use the term "data checksums worker launcher". > Wouldn't "data checksums launcher" (i.e., dropping "worker") be a better name? Fixed. > + <firstterm>data checksums worker launcher</firstterm> and the process > + which operate on the individual databases is known as the > + <firstterm>data checksums worker</firstterm>. > > Shouldn't "the process which operate" be either "the process which operates" > or "the processes which operate"? Since multiple data checksums workers > can run, the latter seems more appropriate. Currently the processing is limited to a single worker, so I think the proposed wording is more appropriate. Expanding the code to handle multiple parallel workers was left as a future enhancement (it will perhaps require better IO throttling etc). > While looking at the related code, I also found that > B_DATACHECKSUMSWORKER_LAUNCHER and B_DATACHECKSUMSWORKER_WORKER > were added as BackendType entries in miscadmin.h. Since those entries appear > under the comment for auxiliary processes, it could give the impression that > the data checksums launcher and workers are auxiliary processes. To avoid > that confusion, would it make sense to add a comment such as: > > ------------------------- > B_WAL_WRITER, > > + /* > + * XXXXXXXXXX > + */ > B_DATACHECKSUMSWORKER_LAUNCHER, > B_DATACHECKSUMSWORKER_WORKER, > ------------------------- Fair point, see attached. > BTW, I'm also wondering why dedicated BackendType entries were added for > the data checksums processes, while other background workers don't have > their own BackendType values. They are need for the pgstat system though aren't they, or am I missing something? -- Daniel Gustafsson Attachments: [application/octet-stream] v2-0002-Add-a-comment-to-distinguish-backend-types.patch (976B, ../../[email protected]/2-v2-0002-Add-a-comment-to-distinguish-backend-types.patch) download | inline diff: From 4c0ba4fd236548abfecdf0d6d242b3a57b2754f7 Mon Sep 17 00:00:00 2001 From: Daniel Gustafsson <[email protected]> Date: Fri, 10 Jul 2026 11:37:06 +0200 Subject: [PATCH v2 2/2] Add a comment to distinguish backend types The data checksums entries were seemingly auxiliary processes from reading the code, but they are in fact background workers. Add a comment to clarify. Reported-by: Fujii Masao <[email protected]> Discussion: https://postgr.es/m/CAHGQGwFsBjQs2fv7b72hxzGV_fJMh6LAg4E83pNfDOu1jVgWCA@mail.gmail.com --- src/include/miscadmin.h | 1 + 1 file changed, 1 insertion(+) diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h index 7170a4bff98..16da60721bf 100644 --- a/src/include/miscadmin.h +++ b/src/include/miscadmin.h @@ -370,6 +370,7 @@ typedef enum BackendType B_WAL_SUMMARIZER, B_WAL_WRITER, + /* Background worker processes */ B_DATACHECKSUMSWORKER_LAUNCHER, B_DATACHECKSUMSWORKER_WORKER, -- 2.39.3 (Apple Git-146) [application/octet-stream] v2-0001-doc-Fix-glossary-entry-for-data-checksums-workers.patch (3.7K, ../../[email protected]/3-v2-0001-doc-Fix-glossary-entry-for-data-checksums-workers.patch) download | inline diff: From efaefdf09522fe2dfaf63347cd96a8ba341f340c Mon Sep 17 00:00:00 2001 From: Daniel Gustafsson <[email protected]> Date: Thu, 9 Jul 2026 10:23:22 +0200 Subject: [PATCH v2 1/2] doc: Fix glossary entry for data checksums workers The glossary entry for data checksums workers incorrectly stated that they were auxiliary processes, but they are implementerd as background workers. Fix, and while there, simplify the entry by combining the worker and launcher into a single glossary term. Reported-by: Fujii Masao <[email protected]> Discussion: https://postgr.es/m/CAHGQGwEv-C9ia+rBYyePzO8F=5FVvS412ZqcOupazuOb5RafNg@mail.gmail.com --- doc/src/sgml/glossary.sgml | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml index f7f69afc922..9052f8bbf41 100644 --- a/doc/src/sgml/glossary.sgml +++ b/doc/src/sgml/glossary.sgml @@ -199,8 +199,6 @@ (but not the autovacuum workers), the <glossterm linkend="glossary-background-writer">background writer</glossterm>, the <glossterm linkend="glossary-checkpointer">checkpointer</glossterm>, - the <glossterm linkend="glossary-data-checksums-worker">data checksums worker</glossterm>, - the <glossterm linkend="glossary-data-checksums-worker-launcher">data checksums worker launcher</glossterm>, the <glossterm linkend="glossary-logger">logger</glossterm>, the <glossterm linkend="glossary-startup-process">startup process</glossterm>, the <glossterm linkend="glossary-wal-archiver">WAL archiver</glossterm>, @@ -236,8 +234,9 @@ which runs system- or user-supplied code. Serves as infrastructure for several features in <productname>PostgreSQL</productname>, such as - <glossterm linkend="glossary-replication">logical replication</glossterm> - and <glossterm linkend="glossary-parallel-query">parallel queries</glossterm>. + <glossterm linkend="glossary-replication">logical replication</glossterm>, + <glossterm linkend="glossary-parallel-query">parallel queries</glossterm> and + <glossterm linkend="glossary-data-checksums-process">data checksums processing</glossterm>. In addition, <glossterm linkend="glossary-extension">Extensions</glossterm> can add custom background worker processes. </para> @@ -588,24 +587,17 @@ <glosssee otherterm="glossary-data-directory" /> </glossentry> - <glossentry id="glossary-data-checksums-worker"> - <glossterm>Data Checksums Worker</glossterm> + <glossentry id="glossary-data-checksums-process"> + <glossterm>Data Checksums (process)</glossterm> <glossdef> <para> - A <glossterm linkend="glossary-background-worker">background worker</glossterm> - which enables data checksums in a specific database. - </para> - </glossdef> - </glossentry> - - <glossentry id="glossary-data-checksums-worker-launcher"> - <glossterm>Data Checksums Worker Launcher</glossterm> - <glossdef> - <para> - A <glossterm linkend="glossary-background-worker">background worker</glossterm> - which starts <glossterm linkend="glossary-data-checksums-worker"> data - checksum worker</glossterm> processes for enabling data checksums in each - database, or disables data checksums cluster-wide. + A set of + <glossterm linkend="glossary-background-worker">background worker</glossterm> + processes which can enable or disable data checksums in a running cluster. + The process which coordinates the work is known as the + <firstterm>data checksums launcher</firstterm> and the process which + operate on the individual databases is known as the + <firstterm>data checksums worker</firstterm>. </para> </glossdef> </glossentry> -- 2.39.3 (Apple Git-146) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2026-07-10 10:00 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-07-08 12:23 Fix data checksum progress reporting docs Fujii Masao <[email protected]> 2026-07-08 13:24 ` Daniel Gustafsson <[email protected]> 2026-07-09 00:17 ` Fujii Masao <[email protected]> 2026-07-09 08:28 ` Daniel Gustafsson <[email protected]> 2026-07-10 01:13 ` Fujii Masao <[email protected]> 2026-07-10 10:00 ` Daniel Gustafsson <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox