MIME-Version: 1.0
References: 
 <CAMm1aWa8p=o60y1Z71bjAjWH8XH2Y+kZ6cNjQ21uK2_i65soOQ@mail.gmail.com>
 <CALj2ACUNnqY7oc5C5yGS1ZbHV1RtQgPGiMkPTm989czCDwag6g@mail.gmail.com>
 <CAEze2Wh_+P9Hs4RQ=tc=70rdB5h0omGx5Kz-BLhsJPRnKQ+cWw@mail.gmail.com>
 <CAMm1aWbuJLEnAjT4_1s8eArmZq=H7oi0Jv=J2GnRpBhkbC_PVw@mail.gmail.com>
 <CAMm1aWYWmqys2O7UekQ0=ZSWSpG9h2qoEW7w46yiWM0eO1Lu7g@mail.gmail.com>
 <CALj2ACXsmvEDsHtADoQeDE-NVyBh4A-D80NmNanTUc37g=FQ-Q@mail.gmail.com>
 <CAMm1aWabDx3V9Hw5L2eB+6EOtr60QWDybZShQsYLCCMNF-es_A@mail.gmail.com>
 <CAMm1aWaFmbc8QCbcPkG7ROV7gjshamS=sJRJd=oghLJM9uOnpQ@mail.gmail.com>
 <20220319001556.atuqbmafvc73pa2k@alap3.anarazel.de>
 <CAMm1aWa_3N1Z2-_YwdNxfogs_rxzpu90VZeKjqwjerCVq9rQAw@mail.gmail.com>
 <20220707000446.hefgyu5xikxwt4md@alap3.anarazel.de>
 <CAMm1aWbfFwT+Cf1eSrXtF2R8ffxXTbH=un8unQ8WbcpcbymnYw@mail.gmail.com>
 <d9546a13-dd86-57d8-3bb4-32136fbb3673@gmail.com>
 <CA+TgmoaJEDirAMq_9q3B8PrXDf1h72S7hhPr4aotakYe9HLoLw@mail.gmail.com>
In-Reply-To: 
 <CA+TgmoaJEDirAMq_9q3B8PrXDf1h72S7hhPr4aotakYe9HLoLw@mail.gmail.com>
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Wed, 16 Nov 2022 16:01:55 +0530
Message-ID: 
 <CALj2ACX5xK_X=MSBh_Qe518O7V5D2phnpXuSD9=-zAkGnyo9vA@mail.gmail.com>
Subject: Re: Report checkpoint progress with pg_stat_progress_checkpoint (was:
 Report checkpoint progress in server logs)
To: Robert Haas <robertmhaas@gmail.com>
Cc: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>,
	Nitin Jadhav <nitinjadhavpostgres@gmail.com>,
 Andres Freund <andres@anarazel.de>,
	Matthias van de Meent <boekewurm+postgres@gmail.com>,
 Ashutosh Sharma <ashu.coek88@gmail.com>,
	Julien Rouhaud <rjuju123@gmail.com>, Bruce Momjian <bruce@momjian.us>,
 Tom Lane <tgl@sss.pgh.pa.us>,
	Magnus Hagander <magnus@hagander.net>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: 
 <https://www.postgresql.org/message-id/CALj2ACX5xK_X%3DMSBh_Qe518O7V5D2phnpXuSD9%3D-zAkGnyo9vA%40mail.gmail.com>
Precedence: bulk

On Wed, Nov 16, 2022 at 1:35 AM Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Fri, Nov 4, 2022 at 4:27 AM Drouvot, Bertrand
> <bertranddrouvot.pg@gmail.com> wrote:
> > Please find attached a rebase in v7.
>
> I don't think it's a good thing that this patch is using the
> progress-reporting machinery. The point of that machinery is that we
> want any backend to be able to report progress for any command it
> happens to be running, and we don't know which command that will be at
> any given point in time, or how many backends will be running any
> given command at once. So we need some generic set of counters that
> can be repurposed for whatever any particular backend happens to be
> doing right at the moment.

Hm.

> But none of that applies to the checkpointer. Any information about
> the checkpointer that we want to expose can just be advertised in a
> dedicated chunk of shared memory, perhaps even by simply adding it to
> CheckpointerShmemStruct. Then you can give the fields whatever names,
> types, and sizes you like, and you don't have to do all of this stuff
> with mapping down to integers and back. The only real disadvantage
> that I can see is then you have to think a bit harder about what the
> concurrency model is here, and maybe you end up reimplementing
> something similar to what the progress-reporting stuff does for you,
> and *maybe* that is a sufficient reason to do it this way.

-1 for CheckpointerShmemStruct as it is being used for running
checkpoints and I don't think adding stats to it is a great idea.
Instead, extending PgStat_CheckpointerStats and using shared memory
stats for reporting progress/last checkpoint related stats is a good
idea IMO. I also think that a new pg_stat_checkpoint view is needed
because, right now, the PgStat_CheckpointerStats stats are exposed via
the pg_stat_bgwriter view, having a separate view for checkpoint stats
is good here. Also, removing CheckpointStatsData and moving all of
those members to PgStat_CheckpointerStats, of course, by being careful
about the amount of shared memory required, is also a good idea IMO.
Going forward, PgStat_CheckpointerStats and pg_stat_checkpoint view
can be a single point of location for all the checkpoint related
stats.

Thoughts?

In fact, I was recently having an off-list chat with Bertrand Drouvot
about the above idea.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com