public inbox for [email protected]help / color / mirror / Atom feed
Re: RFC: pg_stat_logmsg 5+ messages / 3 participants [nested] [flat]
* Re: RFC: pg_stat_logmsg @ 2024-07-18 04:32 Michael Paquier <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Michael Paquier @ 2024-07-18 04:32 UTC (permalink / raw) To: Joe Conway <[email protected]>; +Cc: Tomas Vondra <[email protected]>; Gurjeet Singh <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers On Wed, Jul 17, 2024 at 07:43:13AM -0400, Joe Conway wrote: > On 7/16/24 18:14, Tomas Vondra wrote: >> As for the feature, I've never done a fleet-wide analysis, so if this is >> one of the main use cases, I'm not really sure I can judge if this is a >> good tool for that. It seems like it might be a convenient way to do >> that, but does that require we add the module to contrib? > > I had an offlist chat with Andres about this IIRC and he suggested he > thought it ought to be just built in to the backend as part of the > statistics subsystem. Lately though I have been toying with the idea of > keeping it as an extension and basing it off Michael Paquier's work for > Pluggable cumulative statistics. This may live better as a contrib/ module, serving as well as an extra template for what can be done with the pluggable stats. Adding that in core is of course OK for me if that's the consensus. The APIs for pluggable stats are really the same as what you would store in core, minus the system functions you'd want to add in the catalog .dat files, of course. I'd like to get it this part done by the end of this commit fest to have room with pg_stat_statements for this release, but well, we'll see. As far as I can see everybody who commented on the thread seems kind of OK with the idea to fix the stats kinds IDs in time, like custom RMGRs. That's just simpler implementation-wise, but I'm also looking for more opinions. > Hmm, yeah, I had been planning to include postgres version as part of the > output, but maybe it would need to be part of the key. Seems to me that you should do both, then: add PG_VERSION to the entries, and hash the keys with it for uniqueness. You could also have a reset function that performs a removal of the stats for anything else than the current PG_VERSION, for example. -- Michael Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: RFC: pg_stat_logmsg @ 2026-03-24 08:05 Xuneng Zhou <[email protected]> parent: Michael Paquier <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Xuneng Zhou @ 2026-03-24 08:05 UTC (permalink / raw) To: Michael Paquier <[email protected]>; +Cc: Joe Conway <[email protected]>; Tomas Vondra <[email protected]>; Gurjeet Singh <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers Hi, On Thu, Jul 18, 2024 at 12:33 PM Michael Paquier <[email protected]> wrote: > > On Wed, Jul 17, 2024 at 07:43:13AM -0400, Joe Conway wrote: > > On 7/16/24 18:14, Tomas Vondra wrote: > >> As for the feature, I've never done a fleet-wide analysis, so if this is > >> one of the main use cases, I'm not really sure I can judge if this is a > >> good tool for that. It seems like it might be a convenient way to do > >> that, but does that require we add the module to contrib? > > > > I had an offlist chat with Andres about this IIRC and he suggested he > > thought it ought to be just built in to the backend as part of the > > statistics subsystem. Lately though I have been toying with the idea of > > keeping it as an extension and basing it off Michael Paquier's work for > > Pluggable cumulative statistics. > > This may live better as a contrib/ module, serving as well as an extra > template for what can be done with the pluggable stats. Adding that > in core is of course OK for me if that's the consensus. The APIs for > pluggable stats are really the same as what you would store in core, > minus the system functions you'd want to add in the catalog .dat > files, of course. > > I'd like to get it this part done by the end of this commit fest to > have room with pg_stat_statements for this release, but well, we'll > see. As far as I can see everybody who commented on the thread seems > kind of OK with the idea to fix the stats kinds IDs in time, like > custom RMGRs. That's just simpler implementation-wise, but I'm also > looking for more opinions. > > > Hmm, yeah, I had been planning to include postgres version as part of the > > output, but maybe it would need to be part of the key. > > Seems to me that you should do both, then: add PG_VERSION to the > entries, and hash the keys with it for uniqueness. You could also > have a reset function that performs a removal of the stats for > anything else than the current PG_VERSION, for example. > -- > Michael This feature seems both interesting and useful, yet it hasn't progressed for some time. Looking at the thread history, the patch stalled around 2023 in part because the pluggable cumulative statistics infrastructure was still under development, which offered an opportunity to rework it on top of framework primitives rather than carrying DIY implementations of shared hash tables, persistence, and reset logic. The refactoring opportunity is now basically available. The relevant infrastructure landed in stages: 7949d9594582 (Aug 2024) — kind registration, dshash routing, flush_pending_cb c06e71d1aca (Nov 2024) — write_to_file flag, enabling opt-in persistence 4ba012a8ed9 (Dec 2025) — to_serialized_data / from_serialized_data / finish callbacks I’m considering picking this up and 'd like to move it forward. Before doing so, I am curious about whether there was a consensus on putting it into core or keeping it as an extension, and whether there are any major blockers I might have overlooked. Best, Xuneng ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: RFC: pg_stat_logmsg @ 2026-03-28 15:14 Xuneng Zhou <[email protected]> parent: Xuneng Zhou <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Xuneng Zhou @ 2026-03-28 15:14 UTC (permalink / raw) To: Joe Conway <[email protected]>; +Cc: Michael Paquier <[email protected]>; Tomas Vondra <[email protected]>; Gurjeet Singh <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers On Sat, Mar 28, 2026 at 10:14 PM Joe Conway <[email protected]> wrote: > > On 3/24/26 04:05, Xuneng Zhou wrote: > > Hi, > > > > On Thu, Jul 18, 2024 at 12:33 PM Michael Paquier <[email protected]> wrote: > >> > >> On Wed, Jul 17, 2024 at 07:43:13AM -0400, Joe Conway wrote: > >> > On 7/16/24 18:14, Tomas Vondra wrote: > >> >> As for the feature, I've never done a fleet-wide analysis, so if this is > >> >> one of the main use cases, I'm not really sure I can judge if this is a > >> >> good tool for that. It seems like it might be a convenient way to do > >> >> that, but does that require we add the module to contrib? > >> > > >> > I had an offlist chat with Andres about this IIRC and he suggested he > >> > thought it ought to be just built in to the backend as part of the > >> > statistics subsystem. Lately though I have been toying with the idea of > >> > keeping it as an extension and basing it off Michael Paquier's work for > >> > Pluggable cumulative statistics. > >> > >> This may live better as a contrib/ module, serving as well as an extra > >> template for what can be done with the pluggable stats. Adding that > >> in core is of course OK for me if that's the consensus. The APIs for > >> pluggable stats are really the same as what you would store in core, > >> minus the system functions you'd want to add in the catalog .dat > >> files, of course. > >> > >> I'd like to get it this part done by the end of this commit fest to > >> have room with pg_stat_statements for this release, but well, we'll > >> see. As far as I can see everybody who commented on the thread seems > >> kind of OK with the idea to fix the stats kinds IDs in time, like > >> custom RMGRs. That's just simpler implementation-wise, but I'm also > >> looking for more opinions. > >> > >> > Hmm, yeah, I had been planning to include postgres version as part of the > >> > output, but maybe it would need to be part of the key. > >> > >> Seems to me that you should do both, then: add PG_VERSION to the > >> entries, and hash the keys with it for uniqueness. You could also > >> have a reset function that performs a removal of the stats for > >> anything else than the current PG_VERSION, for example. > >> -- > >> Michael > > > > > > This feature seems both interesting and useful, yet it hasn't > > progressed for some time. Looking at the thread history, the patch > > stalled around 2023 in part because the pluggable cumulative > > statistics infrastructure was still under development, which offered > > an opportunity to rework it on top of framework primitives rather than > > carrying DIY implementations of shared hash tables, persistence, and > > reset logic. > > > > The refactoring opportunity is now basically available. The relevant > > infrastructure landed in stages: > > > > 7949d9594582 (Aug 2024) — kind registration, dshash routing, flush_pending_cb > > c06e71d1aca (Nov 2024) — write_to_file flag, enabling opt-in persistence > > 4ba012a8ed9 (Dec 2025) — to_serialized_data / from_serialized_data / > > finish callbacks > > > > I’m considering picking this up and 'd like to move it forward. Before > > doing so, I am curious about whether there was a consensus on putting > > it into core or keeping it as an extension, and whether there are any > > major blockers I might have overlooked. > > > It has not progressed mainly due to lack of time on my part. It is still > on my TODO list, and I would like to find time to complete it. That would be great! > Personally I would love to see it builtin to core, but an extension > would have the advantage of working with prior releases. In any case, if > it is in core, or even contrib, now is not the right time as it has zero > chance to get in before feature freeze. It would make more sense to > discuss this after April. Yeah, integrating this into core would make it universally accessible, while providing it as an extension would allow earlier versions to benefit. Certainly, it should be considered for upcoming releases. > That said, if you wanted to create an extension on github or whatever > you can of course determine your own timing. > I'll try to create an extension on github. It would be fun to play with and help to learn the new cumulative statistics infra. -- Best, Xuneng ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: RFC: pg_stat_logmsg @ 2026-04-22 14:21 Fabrízio de Royes Mello <[email protected]> parent: Xuneng Zhou <[email protected]> 0 siblings, 1 reply; 5+ messages in thread From: Fabrízio de Royes Mello @ 2026-04-22 14:21 UTC (permalink / raw) To: Xuneng Zhou <[email protected]>; +Cc: Joe Conway <[email protected]>; Michael Paquier <[email protected]>; Tomas Vondra <[email protected]>; Gurjeet Singh <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers On Sat, Mar 28, 2026 at 12:15 PM Xuneng Zhou <[email protected]> wrote: > > On Sat, Mar 28, 2026 at 10:14 PM Joe Conway <[email protected]> wrote: > > > It has not progressed mainly due to lack of time on my part. It is still > > on my TODO list, and I would like to find time to complete it. > > That would be great! > > > Personally I would love to see it builtin to core, but an extension > > would have the advantage of working with prior releases. In any case, if > > it is in core, or even contrib, now is not the right time as it has zero > > chance to get in before feature freeze. It would make more sense to > > discuss this after April. > > Yeah, integrating this into core would make it universally accessible, > while providing it as an extension would allow earlier versions to > benefit. Certainly, it should be considered for upcoming releases. > > > That said, if you wanted to create an extension on github or whatever > > you can of course determine your own timing. > > > > I'll try to create an extension on github. It would be fun to play > with and help to learn the new cumulative statistics infra. > Here's a WIP extension: https://github.com/fabriziomello/pg_stat_log Regards, -- Fabrízio de Royes Mello ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: RFC: pg_stat_logmsg @ 2026-04-23 01:04 Xuneng Zhou <[email protected]> parent: Fabrízio de Royes Mello <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Xuneng Zhou @ 2026-04-23 01:04 UTC (permalink / raw) To: [email protected]; +Cc: Joe Conway <[email protected]>; Michael Paquier <[email protected]>; Tomas Vondra <[email protected]>; Gurjeet Singh <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers Hi Fabrízio, On Wed, Apr 22, 2026 at 10:22 PM Fabrízio de Royes Mello <[email protected]> wrote: > > > On Sat, Mar 28, 2026 at 12:15 PM Xuneng Zhou <[email protected]> wrote: > > > > On Sat, Mar 28, 2026 at 10:14 PM Joe Conway <[email protected]> wrote: > > > > > It has not progressed mainly due to lack of time on my part. It is still > > > on my TODO list, and I would like to find time to complete it. > > > > That would be great! > > > > > Personally I would love to see it builtin to core, but an extension > > > would have the advantage of working with prior releases. In any case, if > > > it is in core, or even contrib, now is not the right time as it has zero > > > chance to get in before feature freeze. It would make more sense to > > > discuss this after April. > > > > Yeah, integrating this into core would make it universally accessible, > > while providing it as an extension would allow earlier versions to > > benefit. Certainly, it should be considered for upcoming releases. > > > > > That said, if you wanted to create an extension on github or whatever > > > you can of course determine your own timing. > > > > > > > I'll try to create an extension on github. It would be fun to play > > with and help to learn the new cumulative statistics infra. > > > > Here's a WIP extension: https://github.com/fabriziomello/pg_stat_log I'll look into it. -- Best, Xuneng ^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2026-04-23 01:04 UTC | newest] Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-07-18 04:32 Re: RFC: pg_stat_logmsg Michael Paquier <[email protected]> 2026-03-24 08:05 ` Xuneng Zhou <[email protected]> 2026-03-28 15:14 ` Xuneng Zhou <[email protected]> 2026-04-22 14:21 ` Fabrízio de Royes Mello <[email protected]> 2026-04-23 01:04 ` Xuneng Zhou <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox