public inbox for [email protected]
help / color / mirror / Atom feedFrom: Nitin Jadhav <[email protected]>
To: Bharath Rupireddy <[email protected]>
Cc: Matthias van de Meent <[email protected]>
Cc: Ashutosh Sharma <[email protected]>
Cc: Julien Rouhaud <[email protected]>
Cc: Bruce Momjian <[email protected]>
Cc: Tom Lane <[email protected]>
Cc: Magnus Hagander <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)
Date: Tue, 8 Mar 2022 20:25:28 +0530
Message-ID: <CAMm1aWaFmbc8QCbcPkG7ROV7gjshamS=sJRJd=oghLJM9uOnpQ@mail.gmail.com> (raw)
In-Reply-To: <CAMm1aWabDx3V9Hw5L2eB+6EOtr60QWDybZShQsYLCCMNF-es_A@mail.gmail.com>
References: <CALj2ACUGWAC56yqhVOisNPXhewfRdeHG4NGTOK_sOStL=7yP+w@mail.gmail.com>
<CAMm1aWY_wThAxmhC_7mkFsYHcNP2_JQLAKJEoY5EcHBi0Q42QQ@mail.gmail.com>
<CAEze2WhfmEWDKcFx8jhcSFhbSoP7ettdYiikNSsVWCNpe5ME0g@mail.gmail.com>
<CAMm1aWYsfgO91mXxy7kzrDcML8nuY3Ab4w9RnSjZ_mr6=Y-hwQ@mail.gmail.com>
<20220217110515.zssjnez4lq2lmrdq@jrouhaud>
<CAMm1aWZ1YY6n8T=0OkwQJJdua7fpkJdKr1fRqfTSrJb1ME9Q5g@mail.gmail.com>
<20220217172724.uhs2v3mgio7y6ga5@jrouhaud>
<CAMm1aWbjzP9g9ebkYqqJvNAW4jh6wP0maKW2Ozcv8PLVu76cFA@mail.gmail.com>
<20220218074351.wgfosjt5lql2vjbr@jrouhaud>
<CAMm1aWadyUhb89DEfgrs2j5i2u--O06x6oTp0EEhgGHtnw-SXA@mail.gmail.com>
<20220219053215.r4wqatln5awklecf@jrouhaud>
<CAMm1aWZzPVpLs7k5sQU4DObvh6YpOhjibcq2ykDrC_n25jeJNA@mail.gmail.com>
<CAMm1aWZ07VYUPpvmV9Ck_VX-o+xH=hEBs-67scd_Us6U6FKbHA@mail.gmail.com>
<CAE9k0Png9tjW5M7wjMKzEoOqkHS-VToP9yC2pr=HX7aQ0yQRNA@mail.gmail.com>
<CAMm1aWa8p=o60y1Z71bjAjWH8XH2Y+kZ6cNjQ21uK2_i65soOQ@mail.gmail.com>
<CALj2ACUNnqY7oc5C5yGS1ZbHV1RtQgPGiMkPTm989czCDwag6g@mail.gmail.com>
<CAEze2Wh_+P9Hs4RQ=tc=70rdB5h0omGx5Kz-BLhsJPRnKQ+cWw@mail.gmail.com>
<CAMm1aWbuJLEnAjT4_1s8eArmZq=H7oi0Jv=J2GnRpBhkbC_PVw@mail.gmail.com>
<CAMm1aWYWmqys2O7UekQ0=ZSWSpG9h2qoEW7w46yiWM0eO1Lu7g@mail.gmail.com>
<CALj2ACXsmvEDsHtADoQeDE-NVyBh4A-D80NmNanTUc37g=FQ-Q@mail.gmail.com>
<CAMm1aWabDx3V9Hw5L2eB+6EOtr60QWDybZShQsYLCCMNF-es_A@mail.gmail.com>
> > 11) Can you be specific what are those "some operations" that forced a
> > checkpoint? May be like, basebackup, createdb or something?
> > + The checkpoint is started because some operation forced a checkpoint.
> >
> I will take care in the next patch.
I feel mentioning/listing the specific operation makes it difficult to
maintain the document. If we add any new functionality in future which
needs a force checkpoint, then there is a high chance that we will
miss to update here. Hence modified it to "The checkpoint is started
because some operation (for which the checkpoint is necessary) is
forced the checkpoint".
Fixed other comments as per the discussion above.
Please find the v5 patch attached and share your thoughts.
Thanks & Regards,
Nitin Jadhav
On Mon, Mar 7, 2022 at 7:45 PM Nitin Jadhav
<[email protected]> wrote:
>
> > 1) Can we convert below into pgstat_progress_update_multi_param, just
> > to avoid function calls?
> > pgstat_progress_update_param(PROGRESS_CHECKPOINT_LSN, checkPoint.redo);
> > pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
> >
> > 2) Why are we not having special phase for CheckPointReplicationOrigin
> > as it does good bunch of work (writing to disk, XLogFlush,
> > durable_rename) especially when max_replication_slots is large?
> >
> > 3) I don't think "requested" is necessary here as it doesn't add any
> > value or it's not a checkpoint kind or such, you can remove it.
> >
> > 4) s:/'recycling old XLOG files'/'recycling old WAL files'
> > + WHEN 16 THEN 'recycling old XLOG files'
> >
> > 5) Can we place CREATE VIEW pg_stat_progress_checkpoint AS definition
> > next to pg_stat_progress_copy in system_view.sql? It looks like all
> > the progress reporting views are next to each other.
>
> I will take care in the next patch.
> ---
>
> > 6) How about shutdown and end-of-recovery checkpoint? Are you planning
> > to have an ereport_startup_progress mechanism as 0002?
>
> I thought of including it earlier then I felt lets first make the
> current patch stable. Once all the fields are properly decided and the
> patch gets in then we can easily extend the functionality to shutdown
> and end-of-recovery cases. I have also observed that the timer
> functionality wont work properly in case of shutdown as we are doing
> an immediate checkpoint. So this needs a lot of discussion and I would
> like to handle this on a separate thread.
> ---
>
> > 7) I think you don't need to call checkpoint_progress_start and
> > pgstat_progress_update_param, any other progress reporting function
> > for shutdown and end-of-recovery checkpoint right?
>
> I had included the guards earlier and then removed later based on the
> discussion upthread.
> ---
>
> > 8) Not for all kinds of checkpoints right? pg_stat_progress_checkpoint
> > can't show progress report for shutdown and end-of-recovery
> > checkpoint, I think you need to specify that here in wal.sgml and
> > checkpoint.sgml.
> > + command <command>CHECKPOINT</command>. The checkpointer process running the
> > + checkpoint will report its progress in the
> > + <structname>pg_stat_progress_checkpoint</structname> view. See
> > + <xref linkend="checkpoint-progress-reporting"/> for details.
> >
> > 9) Can you add a test case for pg_stat_progress_checkpoint view? I
> > think it's good to add one. See, below for reference:
> > -- Add a trigger to catch and print the contents of the catalog view
> > -- pg_stat_progress_copy during data insertion. This allows to test
> > -- the validation of some progress reports for COPY FROM where the trigger
> > -- would fire.
> > create function notice_after_tab_progress_reporting() returns trigger AS
> > $$
> > declare report record;
> >
> > 10) Typo: it's not "is happens"
> > + The checkpoint is happens without delays.
> >
> > 11) Can you be specific what are those "some operations" that forced a
> > checkpoint? May be like, basebackup, createdb or something?
> > + The checkpoint is started because some operation forced a checkpoint.
> >
> > 12) Can you be a bit elobartive here who waits? Something like the
> > backend that requested checkpoint will wait until it's completion ....
> > + Wait for completion before returning.
> >
> > 13) "removing unneeded or flushing needed logical rewrite mapping files"
> > + The checkpointer process is currently removing/flushing the logical
> >
> > 14) "old WAL files"
> > + The checkpointer process is currently recycling old XLOG files.
>
> I will take care in the next patch.
>
> Thanks & Regards,
> Nitin Jadhav
>
> On Wed, Mar 2, 2022 at 11:52 PM Bharath Rupireddy
> <[email protected]> wrote:
> >
> > On Wed, Mar 2, 2022 at 4:45 PM Nitin Jadhav
> > <[email protected]> wrote:
> > > > Also, how about special phases for SyncPostCheckpoint(),
> > > > SyncPreCheckpoint(), InvalidateObsoleteReplicationSlots(),
> > > > PreallocXlogFiles() (it currently pre-allocates only 1 WAL file, but
> > > > it might be increase in future (?)), TruncateSUBTRANS()?
> > >
> > > SyncPreCheckpoint() is just incrementing a counter and
> > > PreallocXlogFiles() currently pre-allocates only 1 WAL file. I feel
> > > there is no need to add any phases for these as of now. We can add in
> > > the future if necessary. Added phases for SyncPostCheckpoint(),
> > > InvalidateObsoleteReplicationSlots() and TruncateSUBTRANS().
> >
> > Okay.
> >
> > > > 10) I'm not sure if it's discussed, how about adding the number of
> > > > snapshot/mapping files so far the checkpoint has processed in file
> > > > processing while loops of
> > > > CheckPointSnapBuild/CheckPointLogicalRewriteHeap? Sometimes, there can
> > > > be many logical snapshot or mapping files and users may be interested
> > > > in knowing the so-far-processed-file-count.
> > >
> > > I had thought about this while sharing the v1 patch and mentioned my
> > > views upthread. I feel it won't give meaningful progress information
> > > (It can be treated as statistics). Hence not included. Thoughts?
> >
> > Okay. If there are any complaints about it we can always add them later.
> >
> > > > > > As mentioned upthread, there can be multiple backends that request a
> > > > > > checkpoint, so unless we want to store an array of pid we should store a number
> > > > > > of backend that are waiting for a new checkpoint.
> > > > >
> > > > > Yeah, you are right. Let's not go that path and store an array of
> > > > > pids. I don't see a strong use-case with the pid of the process
> > > > > requesting checkpoint. If required, we can add it later once the
> > > > > pg_stat_progress_checkpoint view gets in.
> > > >
> > > > I don't think that's really necessary to give the pid list.
> > > >
> > > > If you requested a new checkpoint, it doesn't matter if it's only your backend
> > > > that triggered it, another backend or a few other dozen, the result will be the
> > > > same and you have the information that the request has been seen. We could
> > > > store just a bool for that but having a number instead also gives a bit more
> > > > information and may allow you to detect some broken logic on your client code
> > > > if it keeps increasing.
> > >
> > > It's a good metric to show in the view but the information is not
> > > readily available. Additional code is required to calculate the number
> > > of requests. Is it worth doing that? I feel this can be added later if
> > > required.
> >
> > Yes, we can always add it later if required.
> >
> > > Please find the v4 patch attached and share your thoughts.
> >
> > I reviewed v4 patch, here are my comments:
> >
> > 1) Can we convert below into pgstat_progress_update_multi_param, just
> > to avoid function calls?
> > pgstat_progress_update_param(PROGRESS_CHECKPOINT_LSN, checkPoint.redo);
> > pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
> >
> > 2) Why are we not having special phase for CheckPointReplicationOrigin
> > as it does good bunch of work (writing to disk, XLogFlush,
> > durable_rename) especially when max_replication_slots is large?
> >
> > 3) I don't think "requested" is necessary here as it doesn't add any
> > value or it's not a checkpoint kind or such, you can remove it.
> >
> > 4) s:/'recycling old XLOG files'/'recycling old WAL files'
> > + WHEN 16 THEN 'recycling old XLOG files'
> >
> > 5) Can we place CREATE VIEW pg_stat_progress_checkpoint AS definition
> > next to pg_stat_progress_copy in system_view.sql? It looks like all
> > the progress reporting views are next to each other.
> >
> > 6) How about shutdown and end-of-recovery checkpoint? Are you planning
> > to have an ereport_startup_progress mechanism as 0002?
> >
> > 7) I think you don't need to call checkpoint_progress_start and
> > pgstat_progress_update_param, any other progress reporting function
> > for shutdown and end-of-recovery checkpoint right?
> >
> > 8) Not for all kinds of checkpoints right? pg_stat_progress_checkpoint
> > can't show progress report for shutdown and end-of-recovery
> > checkpoint, I think you need to specify that here in wal.sgml and
> > checkpoint.sgml.
> > + command <command>CHECKPOINT</command>. The checkpointer process running the
> > + checkpoint will report its progress in the
> > + <structname>pg_stat_progress_checkpoint</structname> view. See
> > + <xref linkend="checkpoint-progress-reporting"/> for details.
> >
> > 9) Can you add a test case for pg_stat_progress_checkpoint view? I
> > think it's good to add one. See, below for reference:
> > -- Add a trigger to catch and print the contents of the catalog view
> > -- pg_stat_progress_copy during data insertion. This allows to test
> > -- the validation of some progress reports for COPY FROM where the trigger
> > -- would fire.
> > create function notice_after_tab_progress_reporting() returns trigger AS
> > $$
> > declare report record;
> >
> > 10) Typo: it's not "is happens"
> > + The checkpoint is happens without delays.
> >
> > 11) Can you be specific what are those "some operations" that forced a
> > checkpoint? May be like, basebackup, createdb or something?
> > + The checkpoint is started because some operation forced a checkpoint.
> >
> > 12) Can you be a bit elobartive here who waits? Something like the
> > backend that requested checkpoint will wait until it's completion ....
> > + Wait for completion before returning.
> >
> > 13) "removing unneeded or flushing needed logical rewrite mapping files"
> > + The checkpointer process is currently removing/flushing the logical
> >
> > 14) "old WAL files"
> > + The checkpointer process is currently recycling old XLOG files.
> >
> > Regards,
> > Bharath Rupireddy.
Attachments:
[application/octet-stream] v5-0001-pg_stat_progress_checkpoint-view.patch (38.6K, ../CAMm1aWaFmbc8QCbcPkG7ROV7gjshamS=sJRJd=oghLJM9uOnpQ@mail.gmail.com/2-v5-0001-pg_stat_progress_checkpoint-view.patch)
download | inline diff:
From 8e1ec8bb49dfe7c753e11ac5c97a82751dc1d1b2 Mon Sep 17 00:00:00 2001
From: Nitin Jadhav <[email protected]>
Date: Tue, 8 Mar 2022 14:36:48 +0000
Subject: [PATCH] pg_stat_progress_checkpoint view
---
doc/src/sgml/monitoring.sgml | 404 +++++++++++++++++++++++++-
doc/src/sgml/ref/checkpoint.sgml | 7 +
doc/src/sgml/wal.sgml | 6 +-
src/backend/access/transam/xlog.c | 102 +++++++
src/backend/catalog/system_views.sql | 59 ++++
src/backend/postmaster/checkpointer.c | 4 +
src/backend/storage/buffer/bufmgr.c | 7 +
src/backend/storage/sync/sync.c | 6 +
src/backend/utils/adt/pgstatfuncs.c | 2 +
src/include/commands/progress.h | 38 +++
src/include/utils/backend_progress.h | 3 +-
src/test/regress/expected/rules.out | 106 +++++++
12 files changed, 741 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 9fb62fec8e..18f4cb4221 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -401,6 +401,13 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
See <xref linkend='copy-progress-reporting'/>.
</entry>
</row>
+
+ <row>
+ <entry><structname>pg_stat_progress_checkpoint</structname><indexterm><primary>pg_stat_progress_checkpoint</primary></indexterm></entry>
+ <entry>One row only, showing the progress of the checkpoint.
+ See <xref linkend='checkpoint-progress-reporting'/>.
+ </entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -5556,7 +5563,7 @@ SELECT pg_stat_get_backend_pid(s.backendid) AS pid,
which support progress reporting are <command>ANALYZE</command>,
<command>CLUSTER</command>,
<command>CREATE INDEX</command>, <command>VACUUM</command>,
- <command>COPY</command>,
+ <command>COPY</command>, <command>CHECKPOINT</command>
and <xref linkend="protocol-replication-base-backup"/> (i.e., replication
command that <xref linkend="app-pgbasebackup"/> issues to take
a base backup).
@@ -6844,6 +6851,401 @@ SELECT pg_stat_get_backend_pid(s.backendid) AS pid,
</table>
</sect2>
+ <sect2 id="checkpoint-progress-reporting">
+ <title>Checkpoint Progress Reporting</title>
+
+ <indexterm>
+ <primary>pg_stat_progress_checkpoint</primary>
+ </indexterm>
+
+ <para>
+ Whenever the checkpoint operation is running, the
+ <structname>pg_stat_progress_checkpoint</structname> view will contain a
+ single row indicating the progress of the checkpoint. The tables below
+ describe the information that will be reported and provide information about
+ how to interpret it.
+ </para>
+
+ <table id="pg-stat-progress-checkpoint-view" xreflabel="pg_stat_progress_checkpoint">
+ <title><structname>pg_stat_progress_checkpoint</structname> View</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pid</structfield> <type>integer</type>
+ </para>
+ <para>
+ Process ID of the checkpointer process.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>type</structfield> <type>text</type>
+ </para>
+ <para>
+ Type of the checkpoint. See <xref linkend="checkpoint-types"/>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>flags</structfield> <type>text</type>
+ </para>
+ <para>
+ Flags of the checkpoint. See <xref linkend="checkpoint-flags"/>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>next_flags</structfield> <type>text</type>
+ </para>
+ <para>
+ Flags of the next checkpoint. See <xref linkend="checkpoint-flags"/>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>start_lsn</structfield> <type>text</type>
+ </para>
+ <para>
+ The checkpoint start location.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>start_time</structfield> <type>timestamp with time zone</type>
+ </para>
+ <para>
+ Elapsed time of the checkpoint.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>phase</structfield> <type>text</type>
+ </para>
+ <para>
+ Current processing phase. See <xref linkend="checkpoint-phases"/>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>buffers_total</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Total number of buffers to be written. This is estimated and reported
+ as of the beginning of buffer write operation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>buffers_processed</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of buffers processed. This counter increases when the targeted
+ buffer is processed. This number will eventually become equal to
+ <literal>buffers_total</literal> when the checkpoint is
+ complete.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>buffers_written</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of buffers written. This counter only advances when the targeted
+ buffers is written. Note that some of the buffers are processed but may
+ not required to be written. So this count will always be less than or
+ equal to <literal>buffers_total</literal>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>files_total</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Total number of files to be synced. This is estimated and reported as of
+ the beginning of sync operation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>files_synced</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of files synced. This counter advances when the targeted file is
+ synced. This number will eventually become equal to
+ <literal>files_total</literal> when the checkpoint is complete.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <table id="checkpoint-types">
+ <title>Checkpoint Types</title>
+ <tgroup cols="2">
+ <colspec colname="col1" colwidth="1*"/>
+ <colspec colname="col2" colwidth="2*"/>
+ <thead>
+ <row>
+ <entry>Types</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>checkpoint</literal></entry>
+ <entry>
+ The current operation is checkpoint.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>restartpoint</literal></entry>
+ <entry>
+ The current operation is restartpoint.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <table id="checkpoint-flags">
+ <title>Checkpoint Flags</title>
+ <tgroup cols="2">
+ <colspec colname="col1" colwidth="1*"/>
+ <colspec colname="col2" colwidth="2*"/>
+ <thead>
+ <row>
+ <entry>Flags</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>shutdown</literal></entry>
+ <entry>
+ The checkpoint is for shutdown.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>end-of-recovery</literal></entry>
+ <entry>
+ The checkpoint is for end-of-recovery.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>immediate</literal></entry>
+ <entry>
+ The checkpoint happens without delays.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>force</literal></entry>
+ <entry>
+ The checkpoint is started because some operation (for which the
+ checkpoint is necessary) forced a checkpoint.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>flush all</literal></entry>
+ <entry>
+ The checkpoint flushes all pages, including those belonging to unlogged
+ tables.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>wait</literal></entry>
+ <entry>
+ The operations which requested the checkpoint waits for completion
+ before returning.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>requested</literal></entry>
+ <entry>
+ The checkpoint request has been made.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>wal</literal></entry>
+ <entry>
+ The checkpoint is started because <literal>max_wal_size</literal> is
+ reached.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>time</literal></entry>
+ <entry>
+ The checkpoint is started because <literal>checkpoint_timeout</literal>
+ expired.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <table id="checkpoint-phases">
+ <title>Checkpoint Phases</title>
+ <tgroup cols="2">
+ <colspec colname="col1" colwidth="1*"/>
+ <colspec colname="col2" colwidth="2*"/>
+ <thead>
+ <row>
+ <entry>Phase</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>initializing</literal></entry>
+ <entry>
+ The checkpointer process is preparing to begin the checkpoint operation.
+ This phase is expected to be very brief.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>getting virtual transaction IDs</literal></entry>
+ <entry>
+ The checkpointer process is getting the virtual transaction IDs that
+ are delaying the checkpoint.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>checkpointing replication slots</literal></entry>
+ <entry>
+ The checkpointer process is currently flushing all the replication slots
+ to disk.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>checkpointing logical replication snapshot files</literal></entry>
+ <entry>
+ The checkpointer process is currently removing all the serialized
+ snapshot files that are not required anymore.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>checkpointing logical rewrite mapping files</literal></entry>
+ <entry>
+ The checkpointer process is currently removing unwanted or flushing
+ required logical rewrite mapping files.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>checkpointing replication origin</literal></entry>
+ <entry>
+ The checkpointer process is currently performing a checkpoint of each
+ replication origin's progress with respect to the replayed remote LSN.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>checkpointing commit log pages</literal></entry>
+ <entry>
+ The checkpointer process is currently writing commit log pages to disk.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>checkpointing commit time stamp pages</literal></entry>
+ <entry>
+ The checkpointer process is currently writing commit time stamp pages to
+ disk.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>checkpointing subtransaction pages</literal></entry>
+ <entry>
+ The checkpointer process is currently writing subtransaction pages to disk.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>checkpointing multixact pages</literal></entry>
+ <entry>
+ The checkpointer process is currently writing multixact pages to disk.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>checkpointing predicate lock pages</literal></entry>
+ <entry>
+ The checkpointer process is currently writing predicate lock pages to disk.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>checkpointing buffers</literal></entry>
+ <entry>
+ The checkpointer process is currently writing buffers to disk.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>processing file sync requests</literal></entry>
+ <entry>
+ The checkpointer process is currently processing file sync requests.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>performing two phase checkpoint</literal></entry>
+ <entry>
+ The checkpointer process is currently performing two phase checkpoint.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>performing post checkpoint cleanup</literal></entry>
+ <entry>
+ The checkpointer process is currently performing post checkpoint cleanup.
+ It removes any lingering files that can be safely removed.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>invalidating replication slots</literal></entry>
+ <entry>
+ The checkpointer process is currently invalidating replication slots.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>recycling old WAL files</literal></entry>
+ <entry>
+ The checkpointer process is currently recycling old WAL files.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>truncating subtransactions</literal></entry>
+ <entry>
+ The checkpointer process is currently removing the subtransaction
+ segments.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>finalizing</literal></entry>
+ <entry>
+ The checkpointer process is finalizing the checkpoint operation.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ </sect2>
+
</sect1>
<sect1 id="dynamic-trace">
diff --git a/doc/src/sgml/ref/checkpoint.sgml b/doc/src/sgml/ref/checkpoint.sgml
index 1cebc03d15..f33db50cfc 100644
--- a/doc/src/sgml/ref/checkpoint.sgml
+++ b/doc/src/sgml/ref/checkpoint.sgml
@@ -56,6 +56,13 @@ CHECKPOINT
the <link linkend="predefined-roles-table"><literal>pg_checkpointer</literal></link>
role can call <command>CHECKPOINT</command>.
</para>
+
+ <para>
+ The checkpointer process running the checkpoint will report its progress
+ in the <structname>pg_stat_progress_checkpoint</structname> view except for
+ the shutdown and end-of-recovery cases. See
+ <xref linkend="checkpoint-progress-reporting"/> for details.
+ </para>
</refsect1>
<refsect1>
diff --git a/doc/src/sgml/wal.sgml b/doc/src/sgml/wal.sgml
index 2bb27a8468..8520304abb 100644
--- a/doc/src/sgml/wal.sgml
+++ b/doc/src/sgml/wal.sgml
@@ -530,7 +530,11 @@
adjust the <xref linkend="guc-archive-timeout"/> parameter rather than the
checkpoint parameters.)
It is also possible to force a checkpoint by using the SQL
- command <command>CHECKPOINT</command>.
+ command <command>CHECKPOINT</command>. The checkpointer process running the
+ checkpoint will report its progress in the
+ <structname>pg_stat_progress_checkpoint</structname> view except for the
+ shutdown and end-of-recovery cases. See
+ <xref linkend="checkpoint-progress-reporting"/> for details.
</para>
<para>
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 0d2bd7a357..05dabc6a94 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -65,6 +65,7 @@
#include "catalog/catversion.h"
#include "catalog/pg_control.h"
#include "catalog/pg_database.h"
+#include "commands/progress.h"
#include "common/controldata_utils.h"
#include "executor/instrument.h"
#include "miscadmin.h"
@@ -719,6 +720,8 @@ static void WALInsertLockAcquireExclusive(void);
static void WALInsertLockRelease(void);
static void WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt);
+static void checkpoint_progress_start(int flags, int type);
+
/*
* Insert an XLOG record represented by an already-constructed chain of data
* chunks. This is a low-level routine; to construct the WAL record header
@@ -6419,6 +6422,9 @@ CreateCheckPoint(int flags)
XLogCtl->RedoRecPtr = checkPoint.redo;
SpinLockRelease(&XLogCtl->info_lck);
+ /* Prepare to report progress of the checkpoint. */
+ checkpoint_progress_start(flags, PROGRESS_CHECKPOINT_TYPE_CHECKPOINT);
+
/*
* If enabled, log checkpoint start. We postpone this until now so as not
* to log anything if we decided to skip the checkpoint.
@@ -6501,6 +6507,8 @@ CreateCheckPoint(int flags)
* and we will correctly flush the update below. So we cannot miss any
* xacts we need to wait for.
*/
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_GET_VIRTUAL_TRANSACTION_IDS);
vxids = GetVirtualXIDsDelayingChkpt(&nvxids);
if (nvxids > 0)
{
@@ -6604,6 +6612,8 @@ CreateCheckPoint(int flags)
/*
* Let smgr do post-checkpoint cleanup (eg, deleting old files).
*/
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_POST_CHECKPOINT_CLEANUP);
SyncPostCheckpoint();
/*
@@ -6619,6 +6629,9 @@ CreateCheckPoint(int flags)
*/
XLByteToSeg(RedoRecPtr, _logSegNo, wal_segment_size);
KeepLogSeg(recptr, &_logSegNo);
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_INVALIDATE_REPLI_SLOTS);
+
if (InvalidateObsoleteReplicationSlots(_logSegNo))
{
/*
@@ -6629,6 +6642,8 @@ CreateCheckPoint(int flags)
KeepLogSeg(recptr, &_logSegNo);
}
_logSegNo--;
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_RECYCLE_OLD_XLOG);
RemoveOldXlogFiles(_logSegNo, RedoRecPtr, recptr,
checkPoint.ThisTimeLineID);
@@ -6647,11 +6662,21 @@ CreateCheckPoint(int flags)
* StartupSUBTRANS hasn't been called yet.
*/
if (!RecoveryInProgress())
+ {
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_TRUNCATE_SUBTRANS);
TruncateSUBTRANS(GetOldestTransactionIdConsideredRunning());
+ }
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_FINALIZE);
/* Real work is done; log and update stats. */
LogCheckpointEnd(false);
+ /* Stop reporting progress of the checkpoint. */
+ pgstat_progress_end_command();
+
/* Reset the process title */
update_checkpoint_display(flags, false, true);
@@ -6808,29 +6833,63 @@ static void
CheckPointGuts(XLogRecPtr checkPointRedo, int flags)
{
CheckPointRelationMap();
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_REPLI_SLOTS);
CheckPointReplicationSlots();
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_SNAPSHOTS);
CheckPointSnapBuild();
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_LOGICAL_REWRITE_MAPPINGS);
CheckPointLogicalRewriteHeap();
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_REPLI_ORIGIN);
CheckPointReplicationOrigin();
/* Write out all dirty data in SLRUs and the main buffer pool */
TRACE_POSTGRESQL_BUFFER_CHECKPOINT_START(flags);
CheckpointStats.ckpt_write_t = GetCurrentTimestamp();
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_CLOG_PAGES);
CheckPointCLOG();
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_COMMITTS_PAGES);
CheckPointCommitTs();
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_SUBTRANS_PAGES);
CheckPointSUBTRANS();
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_MULTIXACT_PAGES);
CheckPointMultiXact();
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_PREDICATE_LOCK_PAGES);
CheckPointPredicate();
+
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_BUFFERS);
CheckPointBuffers(flags);
/* Perform all queued up fsyncs */
TRACE_POSTGRESQL_BUFFER_CHECKPOINT_SYNC_START();
CheckpointStats.ckpt_sync_t = GetCurrentTimestamp();
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_SYNC_FILES);
ProcessSyncRequests();
CheckpointStats.ckpt_sync_end_t = GetCurrentTimestamp();
TRACE_POSTGRESQL_BUFFER_CHECKPOINT_DONE();
/* We deliberately delay 2PC checkpointing as long as possible */
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_TWO_PHASE);
CheckPointTwoPhase(checkPointRedo);
}
@@ -6977,6 +7036,9 @@ CreateRestartPoint(int flags)
MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
+ /* Prepare to report progress of the restartpoint. */
+ checkpoint_progress_start(flags, PROGRESS_CHECKPOINT_TYPE_RESTARTPOINT);
+
if (log_checkpoints)
LogCheckpointStart(flags, true);
@@ -7051,6 +7113,9 @@ CreateRestartPoint(int flags)
replayPtr = GetXLogReplayRecPtr(&replayTLI);
endptr = (receivePtr < replayPtr) ? replayPtr : receivePtr;
KeepLogSeg(endptr, &_logSegNo);
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_INVALIDATE_REPLI_SLOTS);
+
if (InvalidateObsoleteReplicationSlots(_logSegNo))
{
/*
@@ -7077,6 +7142,8 @@ CreateRestartPoint(int flags)
if (!RecoveryInProgress())
replayTLI = XLogCtl->InsertTimeLineID;
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_RECYCLE_OLD_XLOG);
RemoveOldXlogFiles(_logSegNo, RedoRecPtr, endptr, replayTLI);
/*
@@ -7093,11 +7160,20 @@ CreateRestartPoint(int flags)
* this because StartupSUBTRANS hasn't been called yet.
*/
if (EnableHotStandby)
+ {
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_TRUNCATE_SUBTRANS);
TruncateSUBTRANS(GetOldestTransactionIdConsideredRunning());
+ }
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_PHASE,
+ PROGRESS_CHECKPOINT_PHASE_FINALIZE);
/* Real work is done; log and update stats. */
LogCheckpointEnd(true);
+ /* Stop reporting progress of the restartpoint. */
+ pgstat_progress_end_command();
+
/* Reset the process title */
update_checkpoint_display(flags, true, true);
@@ -9197,3 +9273,29 @@ SetWalWriterSleeping(bool sleeping)
XLogCtl->WalWriterSleeping = sleeping;
SpinLockRelease(&XLogCtl->info_lck);
}
+
+/*
+ * Start reporting progress of the checkpoint.
+ */
+static void
+checkpoint_progress_start(int flags, int type)
+{
+ const int index[] = {
+ PROGRESS_CHECKPOINT_TYPE,
+ PROGRESS_CHECKPOINT_FLAGS,
+ PROGRESS_CHECKPOINT_LSN,
+ PROGRESS_CHECKPOINT_START_TIMESTAMP,
+ PROGRESS_CHECKPOINT_PHASE
+ };
+ int64 val[5];
+
+ pgstat_progress_start_command(PROGRESS_COMMAND_CHECKPOINT, InvalidOid);
+
+ val[0] = type;
+ val[1] = flags;
+ val[2] = RedoRecPtr;
+ val[3] = CheckpointStats.ckpt_start_t;
+ val[4] = PROGRESS_CHECKPOINT_PHASE_INIT;
+
+ pgstat_progress_update_multi_param(5, index, val);
+}
\ No newline at end of file
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 40b7bca5a9..bd8583b038 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1228,6 +1228,65 @@ CREATE VIEW pg_stat_progress_copy AS
FROM pg_stat_get_progress_info('COPY') AS S
LEFT JOIN pg_database D ON S.datid = D.oid;
+CREATE VIEW pg_stat_progress_checkpoint AS
+ SELECT
+ S.pid AS pid,
+ CASE S.param1 WHEN 1 THEN 'checkpoint'
+ WHEN 2 THEN 'restartpoint'
+ END AS type,
+ ( CASE WHEN (S.param2 & 1) > 0 THEN 'shutdown ' ELSE '' END ||
+ CASE WHEN (S.param2 & 2) > 0 THEN 'end-of-recovery ' ELSE '' END ||
+ CASE WHEN (S.param2 & 4) > 0 THEN 'immediate ' ELSE '' END ||
+ CASE WHEN (S.param2 & 8) > 0 THEN 'force ' ELSE '' END ||
+ CASE WHEN (S.param2 & 16) > 0 THEN 'flush-all ' ELSE '' END ||
+ CASE WHEN (S.param2 & 32) > 0 THEN 'wait ' ELSE '' END ||
+ CASE WHEN (S.param2 & 128) > 0 THEN 'wal ' ELSE '' END ||
+ CASE WHEN (S.param2 & 256) > 0 THEN 'time ' ELSE '' END
+ ) AS flags,
+ ( CASE WHEN (S.param3 & 1) > 0 THEN 'shutdown ' ELSE '' END ||
+ CASE WHEN (S.param3 & 2) > 0 THEN 'end-of-recovery ' ELSE '' END ||
+ CASE WHEN (S.param3 & 4) > 0 THEN 'immediate ' ELSE '' END ||
+ CASE WHEN (S.param3 & 8) > 0 THEN 'force ' ELSE '' END ||
+ CASE WHEN (S.param3 & 16) > 0 THEN 'flush-all ' ELSE '' END ||
+ CASE WHEN (S.param3 & 32) > 0 THEN 'wait ' ELSE '' END ||
+ CASE WHEN (S.param3 & 128) > 0 THEN 'wal ' ELSE '' END ||
+ CASE WHEN (S.param3 & 256) > 0 THEN 'time ' ELSE '' END
+ ) AS next_flags,
+ ( '0/0'::pg_lsn +
+ ((CASE
+ WHEN S.param4 < 0 THEN pow(2::numeric, 64::numeric)::numeric
+ ELSE 0::numeric
+ END) +
+ S.param4::numeric)
+ ) AS start_lsn,
+ to_timestamp(946684800 + (S.param5::float8 / 1000000)) AS start_time,
+ CASE S.param6 WHEN 1 THEN 'initializing'
+ WHEN 2 THEN 'getting virtual transaction IDs'
+ WHEN 3 THEN 'checkpointing replication slots'
+ WHEN 4 THEN 'checkpointing logical replication snapshot files'
+ WHEN 5 THEN 'checkpointing logical rewrite mapping files'
+ WHEN 6 THEN 'checkpointing replication origin'
+ WHEN 7 THEN 'checkpointing commit log pages'
+ WHEN 8 THEN 'checkpointing commit time stamp pages'
+ WHEN 9 THEN 'checkpointing subtransaction pages'
+ WHEN 10 THEN 'checkpointing multixact pages'
+ WHEN 11 THEN 'checkpointing predicate lock pages'
+ WHEN 12 THEN 'checkpointing buffers'
+ WHEN 13 THEN 'processing file sync requests'
+ WHEN 14 THEN 'performing two phase checkpoint'
+ WHEN 15 THEN 'performing post checkpoint cleanup'
+ WHEN 16 THEN 'invalidating replication slots'
+ WHEN 17 THEN 'recycling old WAL files'
+ WHEN 18 THEN 'truncating subtransactions'
+ WHEN 19 THEN 'finalizing'
+ END AS phase,
+ S.param7 AS buffers_total,
+ S.param8 AS buffers_processed,
+ S.param9 AS buffers_written,
+ S.param10 AS files_total,
+ S.param11 AS files_synced
+ FROM pg_stat_get_progress_info('CHECKPOINT') AS S;
+
CREATE VIEW pg_user_mappings AS
SELECT
U.oid AS umid,
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 4488e3a443..9b7441ed40 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
#include "access/xlog.h"
#include "access/xlog_internal.h"
#include "access/xlogrecovery.h"
+#include "commands/progress.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -662,6 +663,9 @@ ImmediateCheckpointRequested(void)
{
volatile CheckpointerShmemStruct *cps = CheckpointerShmem;
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_NEXT_CHECKPOINT_FLAGS,
+ cps->ckpt_flags);
+
/*
* We don't need to acquire the ckpt_lck in this case because we're only
* looking at a single flag bit.
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f5459c68f8..9663035d7a 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -38,6 +38,7 @@
#include "access/xlogutils.h"
#include "catalog/catalog.h"
#include "catalog/storage.h"
+#include "commands/progress.h"
#include "executor/instrument.h"
#include "lib/binaryheap.h"
#include "miscadmin.h"
@@ -2012,6 +2013,8 @@ BufferSync(int flags)
WritebackContextInit(&wb_context, &checkpoint_flush_after);
TRACE_POSTGRESQL_BUFFER_SYNC_START(NBuffers, num_to_scan);
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_BUFFERS_TOTAL,
+ num_to_scan);
/*
* Sort buffers that need to be written to reduce the likelihood of random
@@ -2129,6 +2132,8 @@ BufferSync(int flags)
bufHdr = GetBufferDescriptor(buf_id);
num_processed++;
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_BUFFERS_PROCESSED,
+ num_processed);
/*
* We don't need to acquire the lock here, because we're only looking
@@ -2149,6 +2154,8 @@ BufferSync(int flags)
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
PendingCheckpointerStats.m_buf_written_checkpoints++;
num_written++;
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_BUFFERS_WRITTEN,
+ num_written);
}
}
diff --git a/src/backend/storage/sync/sync.c b/src/backend/storage/sync/sync.c
index e161d57761..638d3eb781 100644
--- a/src/backend/storage/sync/sync.c
+++ b/src/backend/storage/sync/sync.c
@@ -23,6 +23,7 @@
#include "access/multixact.h"
#include "access/xlog.h"
#include "access/xlogutils.h"
+#include "commands/progress.h"
#include "commands/tablespace.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -356,6 +357,9 @@ ProcessSyncRequests(void)
/* Now scan the hashtable for fsync requests to process */
absorb_counter = FSYNCS_PER_ABSORB;
hash_seq_init(&hstat, pendingOps);
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_FILES_TOTAL,
+ hash_get_num_entries(pendingOps));
+
while ((entry = (PendingFsyncEntry *) hash_seq_search(&hstat)) != NULL)
{
int failures;
@@ -419,6 +423,8 @@ ProcessSyncRequests(void)
longest = elapsed;
total_elapsed += elapsed;
processed++;
+ pgstat_progress_update_param(PROGRESS_CHECKPOINT_FILES_SYNCED,
+ processed);
if (log_checkpoints)
elog(DEBUG1, "checkpoint sync: number=%d file=%s time=%.3f ms",
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index fd993d0d5f..95df730415 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -494,6 +494,8 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
cmdtype = PROGRESS_COMMAND_BASEBACKUP;
else if (pg_strcasecmp(cmd, "COPY") == 0)
cmdtype = PROGRESS_COMMAND_COPY;
+ else if (pg_strcasecmp(cmd, "CHECKPOINT") == 0)
+ cmdtype = PROGRESS_COMMAND_CHECKPOINT;
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
diff --git a/src/include/commands/progress.h b/src/include/commands/progress.h
index a28938caf4..316886e6f0 100644
--- a/src/include/commands/progress.h
+++ b/src/include/commands/progress.h
@@ -151,4 +151,42 @@
#define PROGRESS_COPY_TYPE_PIPE 3
#define PROGRESS_COPY_TYPE_CALLBACK 4
+/* Progress parameters for checkpoint */
+#define PROGRESS_CHECKPOINT_TYPE 0
+#define PROGRESS_CHECKPOINT_FLAGS 1
+#define PROGRESS_CHECKPOINT_NEXT_CHECKPOINT_FLAGS 2
+#define PROGRESS_CHECKPOINT_LSN 3
+#define PROGRESS_CHECKPOINT_START_TIMESTAMP 4
+#define PROGRESS_CHECKPOINT_PHASE 5
+#define PROGRESS_CHECKPOINT_BUFFERS_TOTAL 6
+#define PROGRESS_CHECKPOINT_BUFFERS_PROCESSED 7
+#define PROGRESS_CHECKPOINT_BUFFERS_WRITTEN 8
+#define PROGRESS_CHECKPOINT_FILES_TOTAL 9
+#define PROGRESS_CHECKPOINT_FILES_SYNCED 10
+
+/* Types of checkpoint (as advertised via PROGRESS_CHECKPOINT_TYPE) */
+#define PROGRESS_CHECKPOINT_TYPE_CHECKPOINT 1
+#define PROGRESS_CHECKPOINT_TYPE_RESTARTPOINT 2
+
+/* Phases of checkpoint (as advertised via PROGRESS_CHECKPOINT_PHASE) */
+#define PROGRESS_CHECKPOINT_PHASE_INIT 1
+#define PROGRESS_CHECKPOINT_PHASE_GET_VIRTUAL_TRANSACTION_IDS 2
+#define PROGRESS_CHECKPOINT_PHASE_REPLI_SLOTS 3
+#define PROGRESS_CHECKPOINT_PHASE_SNAPSHOTS 4
+#define PROGRESS_CHECKPOINT_PHASE_LOGICAL_REWRITE_MAPPINGS 5
+#define PROGRESS_CHECKPOINT_PHASE_REPLI_ORIGIN 6
+#define PROGRESS_CHECKPOINT_PHASE_CLOG_PAGES 7
+#define PROGRESS_CHECKPOINT_PHASE_COMMITTS_PAGES 8
+#define PROGRESS_CHECKPOINT_PHASE_SUBTRANS_PAGES 9
+#define PROGRESS_CHECKPOINT_PHASE_MULTIXACT_PAGES 10
+#define PROGRESS_CHECKPOINT_PHASE_PREDICATE_LOCK_PAGES 11
+#define PROGRESS_CHECKPOINT_PHASE_BUFFERS 12
+#define PROGRESS_CHECKPOINT_PHASE_SYNC_FILES 13
+#define PROGRESS_CHECKPOINT_PHASE_TWO_PHASE 14
+#define PROGRESS_CHECKPOINT_PHASE_POST_CHECKPOINT_CLEANUP 15
+#define PROGRESS_CHECKPOINT_PHASE_INVALIDATE_REPLI_SLOTS 16
+#define PROGRESS_CHECKPOINT_PHASE_RECYCLE_OLD_XLOG 17
+#define PROGRESS_CHECKPOINT_PHASE_TRUNCATE_SUBTRANS 18
+#define PROGRESS_CHECKPOINT_PHASE_FINALIZE 19
+
#endif
diff --git a/src/include/utils/backend_progress.h b/src/include/utils/backend_progress.h
index 47bf8029b0..02d51fb948 100644
--- a/src/include/utils/backend_progress.h
+++ b/src/include/utils/backend_progress.h
@@ -27,7 +27,8 @@ typedef enum ProgressCommandType
PROGRESS_COMMAND_CLUSTER,
PROGRESS_COMMAND_CREATE_INDEX,
PROGRESS_COMMAND_BASEBACKUP,
- PROGRESS_COMMAND_COPY
+ PROGRESS_COMMAND_COPY,
+ PROGRESS_COMMAND_CHECKPOINT
} ProgressCommandType;
#define PGSTAT_NUM_PROGRESS_PARAM 20
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index ac468568a1..b53a29e372 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1897,6 +1897,112 @@ pg_stat_progress_basebackup| SELECT s.pid,
s.param4 AS tablespaces_total,
s.param5 AS tablespaces_streamed
FROM pg_stat_get_progress_info('BASEBACKUP'::text) s(pid, datid, relid, param1, param2, param3, param4, param5, param6, param7, param8, param9, param10, param11, param12, param13, param14, param15, param16, param17, param18, param19, param20);
+pg_stat_progress_checkpoint| SELECT s.pid,
+ CASE s.param1
+ WHEN 1 THEN 'checkpoint'::text
+ WHEN 2 THEN 'restartpoint'::text
+ ELSE NULL::text
+ END AS type,
+ (((((((
+ CASE
+ WHEN ((s.param2 & (1)::bigint) > 0) THEN 'shutdown '::text
+ ELSE ''::text
+ END ||
+ CASE
+ WHEN ((s.param2 & (2)::bigint) > 0) THEN 'end-of-recovery '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param2 & (4)::bigint) > 0) THEN 'immediate '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param2 & (8)::bigint) > 0) THEN 'force '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param2 & (16)::bigint) > 0) THEN 'flush-all '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param2 & (32)::bigint) > 0) THEN 'wait '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param2 & (128)::bigint) > 0) THEN 'wal '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param2 & (256)::bigint) > 0) THEN 'time '::text
+ ELSE ''::text
+ END) AS flags,
+ (((((((
+ CASE
+ WHEN ((s.param3 & (1)::bigint) > 0) THEN 'shutdown '::text
+ ELSE ''::text
+ END ||
+ CASE
+ WHEN ((s.param3 & (2)::bigint) > 0) THEN 'end-of-recovery '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param3 & (4)::bigint) > 0) THEN 'immediate '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param3 & (8)::bigint) > 0) THEN 'force '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param3 & (16)::bigint) > 0) THEN 'flush-all '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param3 & (32)::bigint) > 0) THEN 'wait '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param3 & (128)::bigint) > 0) THEN 'wal '::text
+ ELSE ''::text
+ END) ||
+ CASE
+ WHEN ((s.param3 & (256)::bigint) > 0) THEN 'time '::text
+ ELSE ''::text
+ END) AS next_flags,
+ ('0/0'::pg_lsn + (
+ CASE
+ WHEN (s.param4 < 0) THEN pow((2)::numeric, (64)::numeric)
+ ELSE (0)::numeric
+ END + (s.param4)::numeric)) AS start_lsn,
+ to_timestamp(((946684800)::double precision + ((s.param5)::double precision / (1000000)::double precision))) AS start_time,
+ CASE s.param6
+ WHEN 1 THEN 'initializing'::text
+ WHEN 2 THEN 'getting virtual transaction IDs'::text
+ WHEN 3 THEN 'checkpointing replication slots'::text
+ WHEN 4 THEN 'checkpointing logical replication snapshot files'::text
+ WHEN 5 THEN 'checkpointing logical rewrite mapping files'::text
+ WHEN 6 THEN 'checkpointing replication origin'::text
+ WHEN 7 THEN 'checkpointing commit log pages'::text
+ WHEN 8 THEN 'checkpointing commit time stamp pages'::text
+ WHEN 9 THEN 'checkpointing subtransaction pages'::text
+ WHEN 10 THEN 'checkpointing multixact pages'::text
+ WHEN 11 THEN 'checkpointing predicate lock pages'::text
+ WHEN 12 THEN 'checkpointing buffers'::text
+ WHEN 13 THEN 'processing file sync requests'::text
+ WHEN 14 THEN 'performing two phase checkpoint'::text
+ WHEN 15 THEN 'performing post checkpoint cleanup'::text
+ WHEN 16 THEN 'invalidating replication slots'::text
+ WHEN 17 THEN 'recycling old WAL files'::text
+ WHEN 18 THEN 'truncating subtransactions'::text
+ WHEN 19 THEN 'finalizing'::text
+ ELSE NULL::text
+ END AS phase,
+ s.param7 AS buffers_total,
+ s.param8 AS buffers_processed,
+ s.param9 AS buffers_written,
+ s.param10 AS files_total,
+ s.param11 AS files_synced
+ FROM pg_stat_get_progress_info('CHECKPOINT'::text) s(pid, datid, relid, param1, param2, param3, param4, param5, param6, param7, param8, param9, param10, param11, param12, param13, param14, param15, param16, param17, param18, param19, param20);
pg_stat_progress_cluster| SELECT s.pid,
s.datid,
d.datname,
--
2.25.1
view thread (74+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)
In-Reply-To: <CAMm1aWaFmbc8QCbcPkG7ROV7gjshamS=sJRJd=oghLJM9uOnpQ@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox