public inbox for [email protected]help / color / mirror / Atom feed
Re: Add checkpoint and redo LSN to LogCheckpointEnd log message 15+ messages / 8 participants [nested] [flat]
* Re: Add checkpoint and redo LSN to LogCheckpointEnd log message @ 2022-01-12 06:09 Julien Rouhaud <[email protected]> 0 siblings, 2 replies; 15+ messages in thread From: Julien Rouhaud @ 2022-01-12 06:09 UTC (permalink / raw) To: Bharath Rupireddy <[email protected]>; +Cc: Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> Hi, On Tue, Dec 28, 2021 at 10:56 AM Bharath Rupireddy <[email protected]> wrote: > > attaching v1-0001-XXX from the initial mail again just for the sake of > completion: Unfortunately this breaks the cfbot as it tries to apply this patch too: http://cfbot.cputube.org/patch_36_3474.log. For this kind of situation I think that the usual solution is to use a .txt extension to make sure that the cfbot won't try to apply it. ^ permalink raw reply [nested|flat] 15+ messages in thread
* cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) @ 2022-01-12 06:19 Tom Lane <[email protected]> parent: Julien Rouhaud <[email protected]> 1 sibling, 1 reply; 15+ messages in thread From: Tom Lane @ 2022-01-12 06:19 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: Julien Rouhaud <[email protected]>; Bharath Rupireddy <[email protected]>; Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> Julien Rouhaud <[email protected]> writes: > For this kind of situation I think that the usual solution is to use a > .txt extension to make sure that the cfbot won't try to apply it. Yeah ... this has come up before. Is there a documented way to attach files that the cfbot will ignore? Two specific scenarios seem to be interesting: 1. You are attaching patch(es) plus some non-patch files 2. You are attaching some random files, and would like to not displace the cfbot's idea of the latest patchset. I don't know exactly how to do either of those. regards, tom lane ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) @ 2022-01-12 06:31 Julien Rouhaud <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 1 reply; 15+ messages in thread From: Julien Rouhaud @ 2022-01-12 06:31 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Thomas Munro <[email protected]>; Bharath Rupireddy <[email protected]>; Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> On Wed, Jan 12, 2022 at 01:19:22AM -0500, Tom Lane wrote: > Julien Rouhaud <[email protected]> writes: > > For this kind of situation I think that the usual solution is to use a > > .txt extension to make sure that the cfbot won't try to apply it. > > Yeah ... this has come up before. Is there a documented way to > attach files that the cfbot will ignore? Not that I know of unfortunately. I think the apply part is done by https://github.com/macdice/cfbot/blob/master/cfbot_patchburner_chroot_ctl.sh#L93-L120, which seems reasonable. So basically any extension outside of those could be used, excluding of course anything clearly suspicious that would be rejected by many email providers. > Two specific scenarios seem to be interesting: > > 1. You are attaching patch(es) plus some non-patch files > > 2. You are attaching some random files, and would like to not > displace the cfbot's idea of the latest patchset. I'm assuming that someone wanting to send an additional patch to be applied on top of the OP patchset is part of 2? ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) @ 2022-01-12 06:37 Tom Lane <[email protected]> parent: Julien Rouhaud <[email protected]> 0 siblings, 2 replies; 15+ messages in thread From: Tom Lane @ 2022-01-12 06:37 UTC (permalink / raw) To: Julien Rouhaud <[email protected]>; +Cc: Thomas Munro <[email protected]>; Bharath Rupireddy <[email protected]>; Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> Julien Rouhaud <[email protected]> writes: > On Wed, Jan 12, 2022 at 01:19:22AM -0500, Tom Lane wrote: >> 2. You are attaching some random files, and would like to not >> displace the cfbot's idea of the latest patchset. > I'm assuming that someone wanting to send an additional patch to be applied on > top of the OP patchset is part of 2? AFAIK, if you're submitting a patch then you have to attach a complete patchset, or the cfbot will be totally lost. Getting the bot to understand incremental patches would be useful for sure ... but it's outside the scope of what I'm asking for now, which is just clear documentation of what the bot can do already. regards, tom lane ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) @ 2022-01-12 06:50 Julien Rouhaud <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 0 replies; 15+ messages in thread From: Julien Rouhaud @ 2022-01-12 06:50 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Thomas Munro <[email protected]>; Bharath Rupireddy <[email protected]>; Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> On Wed, Jan 12, 2022 at 2:37 PM Tom Lane <[email protected]> wrote: > > AFAIK, if you're submitting a patch then you have to attach a complete > patchset, or the cfbot will be totally lost. Getting the bot to > understand incremental patches would be useful for sure ... but it's > outside the scope of what I'm asking for now, which is just clear > documentation of what the bot can do already. Right, but the use case I'm mentioning is a bit different: provide another patch while *not* triggering the cfbot. I've seen cases in the past where people want to share some code to the OP and it seems reasonable to allow that without risking the trigger the cfbot, at least not without the OP validating or adapting the changes. ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) @ 2022-01-12 06:52 Thomas Munro <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 1 reply; 15+ messages in thread From: Thomas Munro @ 2022-01-12 06:52 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Julien Rouhaud <[email protected]>; Bharath Rupireddy <[email protected]>; Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> On Wed, Jan 12, 2022 at 7:37 PM Tom Lane <[email protected]> wrote: > Julien Rouhaud <[email protected]> writes: > > On Wed, Jan 12, 2022 at 01:19:22AM -0500, Tom Lane wrote: > >> 2. You are attaching some random files, and would like to not > >> displace the cfbot's idea of the latest patchset. > > > I'm assuming that someone wanting to send an additional patch to be applied on > > top of the OP patchset is part of 2? > > AFAIK, if you're submitting a patch then you have to attach a complete > patchset, or the cfbot will be totally lost. Getting the bot to > understand incremental patches would be useful for sure ... but it's > outside the scope of what I'm asking for now, which is just clear > documentation of what the bot can do already. By way of documentation, I've just now tried to answer these question in the new FAQ at: https://wiki.postgresql.org/wiki/Cfbot ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) @ 2022-01-12 06:58 Tom Lane <[email protected]> parent: Thomas Munro <[email protected]> 0 siblings, 0 replies; 15+ messages in thread From: Tom Lane @ 2022-01-12 06:58 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: Julien Rouhaud <[email protected]>; Bharath Rupireddy <[email protected]>; Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> Thomas Munro <[email protected]> writes: > By way of documentation, I've just now tried to answer these question > in the new FAQ at: > https://wiki.postgresql.org/wiki/Cfbot Thanks! regards, tom lane ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: Add checkpoint and redo LSN to LogCheckpointEnd log message @ 2022-01-13 06:20 Bharath Rupireddy <[email protected]> parent: Julien Rouhaud <[email protected]> 1 sibling, 1 reply; 15+ messages in thread From: Bharath Rupireddy @ 2022-01-13 06:20 UTC (permalink / raw) To: Julien Rouhaud <[email protected]>; +Cc: Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> On Wed, Jan 12, 2022 at 11:39 AM Julien Rouhaud <[email protected]> wrote: > > Hi, > > On Tue, Dec 28, 2021 at 10:56 AM Bharath Rupireddy > <[email protected]> wrote: > > > > attaching v1-0001-XXX from the initial mail again just for the sake of > > completion: > > Unfortunately this breaks the cfbot as it tries to apply this patch > too: http://cfbot.cputube.org/patch_36_3474.log. > > For this kind of situation I think that the usual solution is to use a > .txt extension to make sure that the cfbot won't try to apply it. Thanks. IMO, the following format of logging is better, so attaching the v2-0001-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.patch as .patch 2021-12-28 02:52:24.464 UTC [2394396] LOG: checkpoint completed at location=0/212FFC8 with REDO start location=0/212FF90: wrote 451 buffers (2.8%); 0 WAL file(s) added, 0 removed, 1 recycled; write=0.012 s, sync=0.032 s, total=0.071 s; sync files=6, longest=0.022 s, average=0.006 s; distance=6272 kB, estimate=6272 kB Others are attached as .txt files. Regards, Bharath Rupireddy. From c23010c67a66dd21e318ae4d475ee0d85c5c1d08 Mon Sep 17 00:00:00 2001 From: Bharath Rupireddy <[email protected]> Date: Tue, 28 Dec 2021 02:52:49 +0000 Subject: [PATCH v2] Add checkpoint and redo LSN to LogCheckpointEnd log message It is useful (for debugging purposes) if the checkpoint end message has the checkpoint LSN and REDO LSN. It gives more context while analyzing checkpoint-related issues. The pg_controldata gives the last checkpoint LSN and REDO LSN, but having this info alongside the log message helps analyze issues that happened previously, connect the dots and identify the root cause. --- src/backend/access/transam/xlog.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 1e1fbe957f..cd3fce6a2c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -8920,11 +8920,14 @@ LogCheckpointEnd(bool restartpoint) if (restartpoint) ereport(LOG, - (errmsg("restartpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("restartpoint completed at location=%X/%X with REDO start location=%X/%X: " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, @@ -8940,11 +8943,14 @@ LogCheckpointEnd(bool restartpoint) (int) (CheckPointDistanceEstimate / 1024.0)))); else ereport(LOG, - (errmsg("checkpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("checkpoint completed at location=%X/%X with REDO start location=%X/%X: " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, -- 2.25.1 From 999cfd53ce4e16ccfff94c0022cd80fe8ff84be5 Mon Sep 17 00:00:00 2001 From: Bharath Rupireddy <[email protected]> Date: Tue, 28 Dec 2021 02:45:51 +0000 Subject: [PATCH v2] Add checkpoint and redo LSN to LogCheckpointEnd log message It is useful (for debugging purposes) if the checkpoint end message has the checkpoint LSN and REDO LSN. It gives more context while analyzing checkpoint-related issues. The pg_controldata gives the last checkpoint LSN and REDO LSN, but having this info alongside the log message helps analyze issues that happened previously, connect the dots and identify the root cause. --- src/backend/access/transam/xlog.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 1e1fbe957f..26eb2a22f3 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -8920,11 +8920,14 @@ LogCheckpointEnd(bool restartpoint) if (restartpoint) ereport(LOG, - (errmsg("restartpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("restartpoint complete: location=%X/%X, REDO start location=%X/%X; " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, @@ -8940,11 +8943,14 @@ LogCheckpointEnd(bool restartpoint) (int) (CheckPointDistanceEstimate / 1024.0)))); else ereport(LOG, - (errmsg("checkpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("checkpoint complete: location=%X/%X, REDO start location=%X/%X; " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, -- 2.25.1 Attachments: [text/plain] v2-0002-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.txt (2.5K, ../../CALj2ACUGutu6As-9p76Ta2bV59J97fbiCc9NSzaWXqxPJdMvag@mail.gmail.com/2-v2-0002-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.txt) download | inline diff: From c23010c67a66dd21e318ae4d475ee0d85c5c1d08 Mon Sep 17 00:00:00 2001 From: Bharath Rupireddy <[email protected]> Date: Tue, 28 Dec 2021 02:52:49 +0000 Subject: [PATCH v2] Add checkpoint and redo LSN to LogCheckpointEnd log message It is useful (for debugging purposes) if the checkpoint end message has the checkpoint LSN and REDO LSN. It gives more context while analyzing checkpoint-related issues. The pg_controldata gives the last checkpoint LSN and REDO LSN, but having this info alongside the log message helps analyze issues that happened previously, connect the dots and identify the root cause. --- src/backend/access/transam/xlog.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 1e1fbe957f..cd3fce6a2c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -8920,11 +8920,14 @@ LogCheckpointEnd(bool restartpoint) if (restartpoint) ereport(LOG, - (errmsg("restartpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("restartpoint completed at location=%X/%X with REDO start location=%X/%X: " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, @@ -8940,11 +8943,14 @@ LogCheckpointEnd(bool restartpoint) (int) (CheckPointDistanceEstimate / 1024.0)))); else ereport(LOG, - (errmsg("checkpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("checkpoint completed at location=%X/%X with REDO start location=%X/%X: " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, -- 2.25.1 [application/octet-stream] v2-0001-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.patch (2.5K, ../../CALj2ACUGutu6As-9p76Ta2bV59J97fbiCc9NSzaWXqxPJdMvag@mail.gmail.com/3-v2-0001-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.patch) download | inline diff: From c23010c67a66dd21e318ae4d475ee0d85c5c1d08 Mon Sep 17 00:00:00 2001 From: Bharath Rupireddy <[email protected]> Date: Tue, 28 Dec 2021 02:52:49 +0000 Subject: [PATCH v2] Add checkpoint and redo LSN to LogCheckpointEnd log message It is useful (for debugging purposes) if the checkpoint end message has the checkpoint LSN and REDO LSN. It gives more context while analyzing checkpoint-related issues. The pg_controldata gives the last checkpoint LSN and REDO LSN, but having this info alongside the log message helps analyze issues that happened previously, connect the dots and identify the root cause. --- src/backend/access/transam/xlog.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 1e1fbe957f..cd3fce6a2c 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -8920,11 +8920,14 @@ LogCheckpointEnd(bool restartpoint) if (restartpoint) ereport(LOG, - (errmsg("restartpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("restartpoint completed at location=%X/%X with REDO start location=%X/%X: " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, @@ -8940,11 +8943,14 @@ LogCheckpointEnd(bool restartpoint) (int) (CheckPointDistanceEstimate / 1024.0)))); else ereport(LOG, - (errmsg("checkpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("checkpoint completed at location=%X/%X with REDO start location=%X/%X: " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, -- 2.25.1 [text/plain] v2-0003-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.txt (2.5K, ../../CALj2ACUGutu6As-9p76Ta2bV59J97fbiCc9NSzaWXqxPJdMvag@mail.gmail.com/4-v2-0003-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.txt) download | inline diff: From 999cfd53ce4e16ccfff94c0022cd80fe8ff84be5 Mon Sep 17 00:00:00 2001 From: Bharath Rupireddy <[email protected]> Date: Tue, 28 Dec 2021 02:45:51 +0000 Subject: [PATCH v2] Add checkpoint and redo LSN to LogCheckpointEnd log message It is useful (for debugging purposes) if the checkpoint end message has the checkpoint LSN and REDO LSN. It gives more context while analyzing checkpoint-related issues. The pg_controldata gives the last checkpoint LSN and REDO LSN, but having this info alongside the log message helps analyze issues that happened previously, connect the dots and identify the root cause. --- src/backend/access/transam/xlog.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 1e1fbe957f..26eb2a22f3 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -8920,11 +8920,14 @@ LogCheckpointEnd(bool restartpoint) if (restartpoint) ereport(LOG, - (errmsg("restartpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("restartpoint complete: location=%X/%X, REDO start location=%X/%X; " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, @@ -8940,11 +8943,14 @@ LogCheckpointEnd(bool restartpoint) (int) (CheckPointDistanceEstimate / 1024.0)))); else ereport(LOG, - (errmsg("checkpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("checkpoint complete: location=%X/%X, REDO start location=%X/%X; " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, -- 2.25.1 ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: Add checkpoint and redo LSN to LogCheckpointEnd log message @ 2022-01-14 13:22 Bharath Rupireddy <[email protected]> parent: Bharath Rupireddy <[email protected]> 0 siblings, 1 reply; 15+ messages in thread From: Bharath Rupireddy @ 2022-01-14 13:22 UTC (permalink / raw) To: Julien Rouhaud <[email protected]>; +Cc: Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> On Thu, Jan 13, 2022 at 11:50 AM Bharath Rupireddy <[email protected]> wrote: > Thanks. IMO, the following format of logging is better, so attaching > the v2-0001-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.patch as > .patch > > 2021-12-28 02:52:24.464 UTC [2394396] LOG: checkpoint completed at > location=0/212FFC8 with REDO start location=0/212FF90: wrote 451 > buffers (2.8%); 0 WAL file(s) added, 0 removed, 1 recycled; > write=0.012 s, sync=0.032 s, total=0.071 s; sync files=6, > longest=0.022 s, average=0.006 s; distance=6272 kB, estimate=6272 kB One of the test cases was failing with the above style of the log message, changing "checkpoint complete" to "checkpoint completed at location" doesn't seem to be a better idea. It looks like the users or the log monitoring tools might be using the same text "checkpoint complete", therefore I don't want to break that. Here's the v3 patch that I think will work better. Please review. Regards, Bharath Rupireddy. Attachments: [application/x-patch] v3-0001-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.patch (2.5K, ../../CALj2ACVM4wjS+wiNZf4ssdNMhSR-XUhR99fYbV=pC9_-=5wCtw@mail.gmail.com/2-v3-0001-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.patch) download | inline diff: From b6f2a53bcaea2d37e317bc3d9922893c10712d3a Mon Sep 17 00:00:00 2001 From: Bharath Rupireddy <[email protected]> Date: Fri, 14 Jan 2022 12:55:30 +0000 Subject: [PATCH v3] Add checkpoint and redo LSN to LogCheckpointEnd log message It is useful (for debugging purposes) if the checkpoint end message has the checkpoint LSN and REDO LSN. It gives more context while analyzing checkpoint-related issues. The pg_controldata gives the last checkpoint LSN and REDO LSN, but having this info alongside the log message helps analyze issues that happened previously, connect the dots and identify the root cause. --- src/backend/access/transam/xlog.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index c9d4cbf3ff..b2eabc72f9 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -8920,11 +8920,14 @@ LogCheckpointEnd(bool restartpoint) if (restartpoint) ereport(LOG, - (errmsg("restartpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("restartpoint complete: location=%X/%X, REDO start location=%X/%X; " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, @@ -8940,11 +8943,14 @@ LogCheckpointEnd(bool restartpoint) (int) (CheckPointDistanceEstimate / 1024.0)))); else ereport(LOG, - (errmsg("checkpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("checkpoint complete: location=%X/%X, REDO start location=%X/%X; " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, -- 2.25.1 ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: Add checkpoint and redo LSN to LogCheckpointEnd log message @ 2022-01-20 00:36 Bossart, Nathan <[email protected]> parent: Bharath Rupireddy <[email protected]> 0 siblings, 2 replies; 15+ messages in thread From: Bossart, Nathan @ 2022-01-20 00:36 UTC (permalink / raw) To: Bharath Rupireddy <[email protected]>; Julien Rouhaud <[email protected]>; +Cc: Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> On 1/3/22, 5:52 PM, "Kyotaro Horiguchi" <[email protected]> wrote: > It seems to me "LSN" or just "location" is more confusing or > mysterious than "REDO LSN" for the average user. If we want to avoid > being technically too detailed, we would use just "start LSN=%X/%X, > end LSN=%X/%X". And it is equivalent to "WAL range=[%X/%X, %X/%X]".. My first instinct was that this should stay aligned with pg_controldata, but that would mean using "location=%X/%X, REDO location=%X/%X," which doesn't seem terribly descriptive. IIUC the "checkpoint location" is the LSN of the WAL record for the checkpoint, and the "checkpoint's REDO location" is the LSN where checkpoint creation began (i.e., what you must retain for crash recovery). My vote is for "start=%X/%X, end=%X/%X." Nathan ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: Add checkpoint and redo LSN to LogCheckpointEnd log message @ 2022-01-20 03:00 Kyotaro Horiguchi <[email protected]> parent: Bossart, Nathan <[email protected]> 1 sibling, 1 reply; 15+ messages in thread From: Kyotaro Horiguchi @ 2022-01-20 03:00 UTC (permalink / raw) To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected] At Thu, 20 Jan 2022 00:36:32 +0000, "Bossart, Nathan" <[email protected]> wrote in > On 1/3/22, 5:52 PM, "Kyotaro Horiguchi" <[email protected]> wrote: > > It seems to me "LSN" or just "location" is more confusing or > > mysterious than "REDO LSN" for the average user. If we want to avoid > > being technically too detailed, we would use just "start LSN=%X/%X, > > end LSN=%X/%X". And it is equivalent to "WAL range=[%X/%X, %X/%X]".. > > My first instinct was that this should stay aligned with > pg_controldata, but that would mean using "location=%X/%X, REDO > location=%X/%X," which doesn't seem terribly descriptive. IIUC the > "checkpoint location" is the LSN of the WAL record for the checkpoint, > and the "checkpoint's REDO location" is the LSN where checkpoint > creation began (i.e., what you must retain for crash recovery). My > vote is for "start=%X/%X, end=%X/%X." +1. Works for me. %X/%X itself expresses it is an LSN. regards. -- Kyotaro Horiguchi NTT Open Source Software Center ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: Add checkpoint and redo LSN to LogCheckpointEnd log message @ 2022-01-27 15:07 Bharath Rupireddy <[email protected]> parent: Bossart, Nathan <[email protected]> 1 sibling, 1 reply; 15+ messages in thread From: Bharath Rupireddy @ 2022-01-27 15:07 UTC (permalink / raw) To: Bossart, Nathan <[email protected]>; +Cc: Julien Rouhaud <[email protected]>; Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> On Thu, Jan 20, 2022 at 6:06 AM Bossart, Nathan <[email protected]> wrote: > > On 1/3/22, 5:52 PM, "Kyotaro Horiguchi" <[email protected]> wrote: > > It seems to me "LSN" or just "location" is more confusing or > > mysterious than "REDO LSN" for the average user. If we want to avoid > > being technically too detailed, we would use just "start LSN=%X/%X, > > end LSN=%X/%X". And it is equivalent to "WAL range=[%X/%X, %X/%X]".. > > My first instinct was that this should stay aligned with > pg_controldata, but that would mean using "location=%X/%X, REDO > location=%X/%X," which doesn't seem terribly descriptive. IIUC the > "checkpoint location" is the LSN of the WAL record for the checkpoint, > and the "checkpoint's REDO location" is the LSN where checkpoint > creation began (i.e., what you must retain for crash recovery). My > vote is for "start=%X/%X, end=%X/%X." I'm still not clear how the REDO location can be treated as a start LSN? Can someone throw some light one what this checkpoint's REDO location is? Regards, Bharath Rupireddy. ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: Add checkpoint and redo LSN to LogCheckpointEnd log message @ 2022-01-27 23:57 Nathan Bossart <[email protected]> parent: Bharath Rupireddy <[email protected]> 0 siblings, 0 replies; 15+ messages in thread From: Nathan Bossart @ 2022-01-27 23:57 UTC (permalink / raw) To: Bharath Rupireddy <[email protected]>; +Cc: Bossart, Nathan <[email protected]>; Julien Rouhaud <[email protected]>; Michael Paquier <[email protected]>; Kyotaro Horiguchi <[email protected]>; PostgreSQL Hackers <[email protected]> On Thu, Jan 27, 2022 at 08:37:37PM +0530, Bharath Rupireddy wrote: > I'm still not clear how the REDO location can be treated as a start > LSN? Can someone throw some light one what this checkpoint's REDO > location is? It's the WAL insert location at the time the checkpoint began (i.e., where you need to begin replaying WAL from after a crash). -- Nathan Bossart Amazon Web Services: https://aws.amazon.com/ ^ permalink raw reply [nested|flat] 15+ messages in thread
* Re: Add checkpoint and redo LSN to LogCheckpointEnd log message @ 2022-01-28 03:13 Bharath Rupireddy <[email protected]> parent: Kyotaro Horiguchi <[email protected]> 0 siblings, 0 replies; 15+ messages in thread From: Bharath Rupireddy @ 2022-01-28 03:13 UTC (permalink / raw) To: Kyotaro Horiguchi <[email protected]>; +Cc: Bossart, Nathan <[email protected]>; Julien Rouhaud <[email protected]>; Michael Paquier <[email protected]>; PostgreSQL Hackers <[email protected]> On Thu, Jan 20, 2022 at 8:30 AM Kyotaro Horiguchi <[email protected]> wrote: > > At Thu, 20 Jan 2022 00:36:32 +0000, "Bossart, Nathan" <[email protected]> wrote in > > On 1/3/22, 5:52 PM, "Kyotaro Horiguchi" <[email protected]> wrote: > > > It seems to me "LSN" or just "location" is more confusing or > > > mysterious than "REDO LSN" for the average user. If we want to avoid > > > being technically too detailed, we would use just "start LSN=%X/%X, > > > end LSN=%X/%X". And it is equivalent to "WAL range=[%X/%X, %X/%X]".. > > > > My first instinct was that this should stay aligned with > > pg_controldata, but that would mean using "location=%X/%X, REDO > > location=%X/%X," which doesn't seem terribly descriptive. IIUC the > > "checkpoint location" is the LSN of the WAL record for the checkpoint, > > and the "checkpoint's REDO location" is the LSN where checkpoint > > creation began (i.e., what you must retain for crash recovery). My > > vote is for "start=%X/%X, end=%X/%X." > > +1. Works for me. %X/%X itself expresses it is an LSN. Thanks for the review. Here's the v4 patch, please have a look. Let's not attempt to change how pg_controldata (tool and core functions) emit the start and end LSN as checkpoint_lsn/redo_lsn and checkpoint location/checkpoint's REDO location. [1] 2022-01-28 03:06:10.213 UTC [2409486] LOG: checkpoint starting: immediate force wait 2022-01-28 03:06:10.257 UTC [2409486] LOG: checkpoint complete: start=0/14D9510, end=0/14D9548; wrote 4 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.007 s, sync=0.008 s, total=0.044 s; sync files=3, longest=0.005 s, average=0.003 s; distance=0 kB, estimate=0 kB 2022-01-28 03:06:42.254 UTC [2409486] LOG: checkpoint starting: immediate force wait 2022-01-28 03:06:42.279 UTC [2409486] LOG: checkpoint complete: start=0/14DBDB8, end=0/14DBDF0; wrote 2 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.004 s, sync=0.004 s, total=0.025 s; sync files=2, longest=0.003 s, average=0.002 s; distance=10 kB, estimate=10 kB Regards, Bharath Rupireddy. Attachments: [application/octet-stream] v4-0001-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.patch (2.4K, ../../CALj2ACUaGJQVbQ5Hms_C2tu+6iJX-EWkEdOiXwX1WfrxQX8pHA@mail.gmail.com/2-v4-0001-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-l.patch) download | inline diff: From f2319df1eba9600c8628910c509757ce5208874f Mon Sep 17 00:00:00 2001 From: Bharath Rupireddy <[email protected]> Date: Fri, 28 Jan 2022 03:07:07 +0000 Subject: [PATCH v4] Add checkpoint and redo LSN to LogCheckpointEnd log message It is useful (for debugging purposes) if the checkpoint end message has the checkpoint LSN(end) and REDO LSN(start). It gives more context while analyzing checkpoint-related issues. The pg_controldata gives the last checkpoint LSN and REDO LSN, but having this info alongside the log message helps analyze issues that happened previously, connect the dots and identify the root cause. --- src/backend/access/transam/xlog.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index dfe2a0bcce..adecc979c2 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -8919,11 +8919,14 @@ LogCheckpointEnd(bool restartpoint) if (restartpoint) ereport(LOG, - (errmsg("restartpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("restartpoint complete: start=%X/%X, end=%X/%X; " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), + LSN_FORMAT_ARGS(ControlFile->checkPoint), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, @@ -8939,11 +8942,14 @@ LogCheckpointEnd(bool restartpoint) (int) (CheckPointDistanceEstimate / 1024.0)))); else ereport(LOG, - (errmsg("checkpoint complete: wrote %d buffers (%.1f%%); " + (errmsg("checkpoint complete: start=%X/%X, end=%X/%X; " + "wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " "distance=%d kB, estimate=%d kB", + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo), + LSN_FORMAT_ARGS(ControlFile->checkPoint), CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, -- 2.25.1 ^ permalink raw reply [nested|flat] 15+ messages in thread
* [PATCH] Rework lwlocknames.txt to become lwlocklist.h @ 2024-03-01 12:03 Alvaro Herrera <[email protected]> 0 siblings, 0 replies; 15+ messages in thread From: Alvaro Herrera @ 2024-03-01 12:03 UTC (permalink / raw) This saves some code and some space in BuiltinTrancheNames, as foreseen in commit 74a730631065. --- src/backend/Makefile | 4 +- src/backend/storage/lmgr/.gitignore | 1 - src/backend/storage/lmgr/Makefile | 9 +- .../storage/lmgr/generate-lwlocknames.pl | 52 ++++++------ src/backend/storage/lmgr/lwlock.c | 15 ++-- src/backend/storage/lmgr/lwlocknames.txt | 60 -------------- .../utils/activity/wait_event_names.txt | 8 +- src/include/storage/lwlocklist.h | 82 +++++++++++++++++++ 8 files changed, 124 insertions(+), 107 deletions(-) delete mode 100644 src/backend/storage/lmgr/lwlocknames.txt create mode 100644 src/include/storage/lwlocklist.h diff --git a/src/backend/Makefile b/src/backend/Makefile index d66e2a4b9f..34bb6393c4 100644 --- a/src/backend/Makefile +++ b/src/backend/Makefile @@ -110,8 +110,8 @@ $(top_builddir)/src/port/libpgport_srv.a: | submake-libpgport parser/gram.h: parser/gram.y $(MAKE) -C parser gram.h -storage/lmgr/lwlocknames.h: storage/lmgr/generate-lwlocknames.pl storage/lmgr/lwlocknames.txt utils/activity/wait_event_names.txt - $(MAKE) -C storage/lmgr lwlocknames.h lwlocknames.c +storage/lmgr/lwlocknames.h: storage/lmgr/generate-lwlocknames.pl $(top_srcdir)/src/include/storage/lwlocklist.h utils/activity/wait_event_names.txt + $(MAKE) -C storage/lmgr lwlocknames.h utils/activity/wait_event_types.h: utils/activity/generate-wait_event_types.pl utils/activity/wait_event_names.txt $(MAKE) -C utils/activity wait_event_types.h pgstat_wait_event.c wait_event_funcs_data.c diff --git a/src/backend/storage/lmgr/.gitignore b/src/backend/storage/lmgr/.gitignore index dab4c3f580..8e5b734f15 100644 --- a/src/backend/storage/lmgr/.gitignore +++ b/src/backend/storage/lmgr/.gitignore @@ -1,3 +1,2 @@ -/lwlocknames.c /lwlocknames.h /s_lock_test diff --git a/src/backend/storage/lmgr/Makefile b/src/backend/storage/lmgr/Makefile index 1aef423384..9d7dbe5abd 100644 --- a/src/backend/storage/lmgr/Makefile +++ b/src/backend/storage/lmgr/Makefile @@ -18,7 +18,6 @@ OBJS = \ lmgr.o \ lock.o \ lwlock.o \ - lwlocknames.o \ predicate.o \ proc.o \ s_lock.o \ @@ -35,11 +34,7 @@ s_lock_test: s_lock.c $(top_builddir)/src/common/libpgcommon.a $(top_builddir)/s $(TASPATH) -L $(top_builddir)/src/common -lpgcommon \ -L $(top_builddir)/src/port -lpgport -lm -o s_lock_test -# see notes in src/backend/parser/Makefile -lwlocknames.c: lwlocknames.h - touch $@ - -lwlocknames.h: $(top_srcdir)/src/backend/storage/lmgr/lwlocknames.txt $(top_srcdir)/src/backend/utils/activity/wait_event_names.txt generate-lwlocknames.pl +lwlocknames.h: $(top_srcdir)/src/include/storage/lwlocklist.h $(top_srcdir)/src/backend/utils/activity/wait_event_names.txt generate-lwlocknames.pl $(PERL) $(srcdir)/generate-lwlocknames.pl $^ check: s_lock_test @@ -47,4 +42,4 @@ check: s_lock_test clean: rm -f s_lock_test - rm -f lwlocknames.h lwlocknames.c + rm -f lwlocknames.h diff --git a/src/backend/storage/lmgr/generate-lwlocknames.pl b/src/backend/storage/lmgr/generate-lwlocknames.pl index 7b93ecf6c1..f46634a180 100644 --- a/src/backend/storage/lmgr/generate-lwlocknames.pl +++ b/src/backend/storage/lmgr/generate-lwlocknames.pl @@ -1,6 +1,6 @@ #!/usr/bin/perl # -# Generate lwlocknames.h and lwlocknames.c from lwlocknames.txt +# Generate lwlocknames.h from lwlocklist.h # Copyright (c) 2000-2024, PostgreSQL Global Development Group use strict; @@ -14,26 +14,22 @@ my $continue = "\n"; GetOptions('outdir:s' => \$output_path); -open my $lwlocknames, '<', $ARGV[0] or die; +open my $lwlocklist, '<', $ARGV[0] or die; open my $wait_event_names, '<', $ARGV[1] or die; # Include PID in suffix in case parallel make runs this multiple times. my $htmp = "$output_path/lwlocknames.h.tmp$$"; -my $ctmp = "$output_path/lwlocknames.c.tmp$$"; open my $h, '>', $htmp or die "Could not open $htmp: $!"; -open my $c, '>', $ctmp or die "Could not open $ctmp: $!"; my $autogen = - "/* autogenerated from src/backend/storage/lmgr/lwlocknames.txt, do not edit */\n"; + "/* autogenerated from src/backend/storage/lmgr/lwlocklist.h, do not edit */\n"; print $h $autogen; print $h "/* there is deliberately not an #ifndef LWLOCKNAMES_H here */\n\n"; -print $c $autogen, "\n"; -print $c "const char *const IndividualLWLockNames[] = {"; # # First, record the predefined LWLocks listed in wait_event_names.txt. We'll -# cross-check those with the ones in lwlocknames.txt. +# cross-check those with the ones in lwlocklist.h. # my @wait_event_lwlocks; my $record_lwlocks = 0; @@ -64,17 +60,30 @@ while (<$wait_event_names>) push(@wait_event_lwlocks, $waiteventname . "Lock"); } +my $in_comment = 0; my $i = 0; -while (<$lwlocknames>) +while (<$lwlocklist>) { chomp; - # Skip comments - next if /^#/; + # Skip uniline C comments and empty lines + next if m{^\s*/\*.*\*/$}; next if /^\s*$/; - die "unable to parse lwlocknames.txt" - unless /^(\w+)\s+(\d+)$/; + # skip multiline C comments + if ($in_comment == 1) + { + $in_comment = 0 if m{\*/}; + next; + } + elsif (m{^\s*/\*}) + { + $in_comment = 1; + next; + } + + die "unable to parse lwlocklist.h line \"$_\"" + unless /^PG_LWLOCK\((\w+),\s+(\d+)\)$/; (my $lockname, my $lockidx) = ($1, $2); @@ -82,25 +91,23 @@ while (<$lwlocknames>) $trimmedlockname =~ s/Lock$//; die "lock names must end with 'Lock'" if $trimmedlockname eq $lockname; - die "lwlocknames.txt not in order" if $lockidx < $lastlockidx; - die "lwlocknames.txt has duplicates" if $lockidx == $lastlockidx; + die "lwlocklist.h not in order" if $lockidx < $lastlockidx; + die "lwlocklist.h has duplicates" if $lockidx == $lastlockidx; - die "$lockname defined in lwlocknames.txt but missing from " + die "$lockname defined in lwlocklist.h but missing from " . "wait_event_names.txt" if $i >= scalar @wait_event_lwlocks; die "lists of predefined LWLocks do not match (first mismatch at " . "$wait_event_lwlocks[$i] in wait_event_names.txt and $lockname in " - . "lwlocknames.txt)" + . "lwlocklist.h)" if $wait_event_lwlocks[$i] ne $lockname; $i++; while ($lastlockidx < $lockidx - 1) { ++$lastlockidx; - printf $c "%s \"<unassigned:%d>\"", $continue, $lastlockidx; $continue = ",\n"; } - printf $c "%s \"%s\"", $continue, $trimmedlockname; $lastlockidx = $lockidx; $continue = ",\n"; @@ -109,18 +116,15 @@ while (<$lwlocknames>) die "$wait_event_lwlocks[$i] defined in wait_event_names.txt but missing from " - . "lwlocknames.txt" + . "lwlocklist.h" if $i < scalar @wait_event_lwlocks; -printf $c "\n};\n"; print $h "\n"; printf $h "#define NUM_INDIVIDUAL_LWLOCKS %s\n", $lastlockidx + 1; close $h; -close $c; rename($htmp, "$output_path/lwlocknames.h") || die "rename: $htmp to $output_path/lwlocknames.h: $!"; -rename($ctmp, "$output_path/lwlocknames.c") || die "rename: $ctmp: $!"; -close $lwlocknames; +close $lwlocklist; diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index d405c61b21..1112389af5 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -115,8 +115,8 @@ StaticAssertDecl(LW_VAL_EXCLUSIVE > (uint32) MAX_BACKENDS, * There are three sorts of LWLock "tranches": * * 1. The individually-named locks defined in lwlocknames.h each have their - * own tranche. The names of these tranches appear in IndividualLWLockNames[] - * in lwlocknames.c. + * own tranche. We absorb the names of these tranches from there into + * BuiltinTrancheNames here. * * 2. There are some predefined tranches for built-in groups of locks. * These are listed in enum BuiltinTrancheIds in lwlock.h, and their names @@ -129,9 +129,10 @@ StaticAssertDecl(LW_VAL_EXCLUSIVE > (uint32) MAX_BACKENDS, * All these names are user-visible as wait event names, so choose with care * ... and do not forget to update the documentation's list of wait events. */ -extern const char *const IndividualLWLockNames[]; /* in lwlocknames.c */ - static const char *const BuiltinTrancheNames[] = { +#define PG_LWLOCK(lockname, id) [id] = CppAsString(lockname), +#include "storage/lwlocknames.h" +#undef PG_LWLOCK [LWTRANCHE_XACT_BUFFER] = "XactBuffer", [LWTRANCHE_COMMITTS_BUFFER] = "CommitTsBuffer", [LWTRANCHE_SUBTRANS_BUFFER] = "SubtransBuffer", @@ -745,11 +746,7 @@ LWLockReportWaitEnd(void) static const char * GetLWTrancheName(uint16 trancheId) { - /* Individual LWLock? */ - if (trancheId < NUM_INDIVIDUAL_LWLOCKS) - return IndividualLWLockNames[trancheId]; - - /* Built-in tranche? */ + /* Built-in tranche or individual LWLock? */ if (trancheId < LWTRANCHE_FIRST_USER_DEFINED) return BuiltinTrancheNames[trancheId]; diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt deleted file mode 100644 index 284d168f77..0000000000 --- a/src/backend/storage/lmgr/lwlocknames.txt +++ /dev/null @@ -1,60 +0,0 @@ -# Some commonly-used locks have predefined positions within MainLWLockArray; -# these are defined here. If you add a lock, add it to the end to avoid -# renumbering the existing locks; if you remove a lock, consider leaving a gap -# in the numbering sequence for the benefit of DTrace and other external -# debugging scripts. Also, do not forget to update the section -# WaitEventLWLock of src/backend/utils/activity/wait_event_names.txt. - -# 0 is available; was formerly BufFreelistLock -ShmemIndexLock 1 -OidGenLock 2 -XidGenLock 3 -ProcArrayLock 4 -SInvalReadLock 5 -SInvalWriteLock 6 -WALBufMappingLock 7 -WALWriteLock 8 -ControlFileLock 9 -# 10 was CheckpointLock -# 11 was XactSLRULock -# 12 was SubtransSLRULock -MultiXactGenLock 13 -# 14 was MultiXactOffsetSLRULock -# 15 was MultiXactMemberSLRULock -RelCacheInitLock 16 -CheckpointerCommLock 17 -TwoPhaseStateLock 18 -TablespaceCreateLock 19 -BtreeVacuumLock 20 -AddinShmemInitLock 21 -AutovacuumLock 22 -AutovacuumScheduleLock 23 -SyncScanLock 24 -RelationMappingLock 25 -#26 was NotifySLRULock -NotifyQueueLock 27 -SerializableXactHashLock 28 -SerializableFinishedListLock 29 -SerializablePredicateListLock 30 -# 31 was SerialSLRULock -SyncRepLock 32 -BackgroundWorkerLock 33 -DynamicSharedMemoryControlLock 34 -AutoFileLock 35 -ReplicationSlotAllocationLock 36 -ReplicationSlotControlLock 37 -#38 was CommitTsSLRULock -CommitTsLock 39 -ReplicationOriginLock 40 -MultiXactTruncationLock 41 -# 42 was OldSnapshotTimeMapLock -LogicalRepWorkerLock 43 -XactTruncationLock 44 -# 45 was XactTruncationLock until removal of BackendRandomLock -WrapLimitsVacuumLock 46 -NotifyQueueTailLock 47 -WaitEventExtensionLock 48 -WALSummarizerLock 49 -DSMRegistryLock 50 -InjectionPointLock 51 -SerialControlLock 52 diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt index ec2f31f82a..105ca977a5 100644 --- a/src/backend/utils/activity/wait_event_names.txt +++ b/src/backend/utils/activity/wait_event_names.txt @@ -279,9 +279,9 @@ Extension "Waiting in an extension." # This class of wait events has its own set of C structure, so these are # only used for the documentation. # -# NB: Predefined LWLocks (i.e., those declared in lwlocknames.txt) must be +# NB: Predefined LWLocks (i.e., those declared in lwlocknames.h) must be # listed in the top section of locks and must be listed in the same order as in -# lwlocknames.txt. +# lwlocknames.h. # Section: ClassName - WaitEventLWLock @@ -332,9 +332,9 @@ SerialControl "Waiting to read or update shared <filename>pg_serial</filename> s # # END OF PREDEFINED LWLOCKS (DO NOT CHANGE THIS LINE) # -# Predefined LWLocks (i.e., those declared in lwlocknames.txt) must be listed +# Predefined LWLocks (i.e., those declared in lwlocknames.h) must be listed # in the section above and must be listed in the same order as in -# lwlocknames.txt. Other LWLocks must be listed in the section below. +# lwlocknames.h. Other LWLocks must be listed in the section below. # XactBuffer "Waiting for I/O on a transaction status SLRU buffer." diff --git a/src/include/storage/lwlocklist.h b/src/include/storage/lwlocklist.h new file mode 100644 index 0000000000..3396eed58e --- /dev/null +++ b/src/include/storage/lwlocklist.h @@ -0,0 +1,82 @@ +/*------------------------------------------------------------------------- + * + * lwlocklist.h + * + * The predefined LWLock list is kept in its own source file for use by + * automatic tools. The exact representation of a keyword is determined by + * the PG_LWLOCK macro, which is not defined in this file; it can be + * defined by the caller for special purposes. + * + * Also, generate-lwlock-names.pl processes this file. + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * IDENTIFICATION + * src/include/storage/lwlocklist.h + * + *------------------------------------------------------------------------- + */ + +/* + * Some commonly-used locks have predefined positions within MainLWLockArray; + * these are defined here. If you add a lock, add it to the end to avoid + * renumbering the existing locks; if you remove a lock, consider leaving a gap + * in the numbering sequence for the benefit of DTrace and other external + * debugging scripts. Also, do not forget to update the section + * WaitEventLWLock of src/backend/utils/activity/wait_event_names.txt. + */ + +/* 0 is available; was formerly BufFreelistLock */ +PG_LWLOCK(ShmemIndexLock, 1) +PG_LWLOCK(OidGenLock, 2) +PG_LWLOCK(XidGenLock, 3) +PG_LWLOCK(ProcArrayLock, 4) +PG_LWLOCK(SInvalReadLock, 5) +PG_LWLOCK(SInvalWriteLock, 6) +PG_LWLOCK(WALBufMappingLock, 7) +PG_LWLOCK(WALWriteLock, 8) +PG_LWLOCK(ControlFileLock, 9) +/* 10 was CheckpointLock */ +/* 11 was XactSLRULock */ +/* 12 was SubtransSLRULock */ +PG_LWLOCK(MultiXactGenLock, 13) +/* 14 was MultiXactOffsetSLRULock */ +/* 15 was MultiXactMemberSLRULock */ +PG_LWLOCK(RelCacheInitLock, 16) +PG_LWLOCK(CheckpointerCommLock, 17) +PG_LWLOCK(TwoPhaseStateLock, 18) +PG_LWLOCK(TablespaceCreateLock, 19) +PG_LWLOCK(BtreeVacuumLock, 20) +PG_LWLOCK(AddinShmemInitLock, 21) +PG_LWLOCK(AutovacuumLock, 22) +PG_LWLOCK(AutovacuumScheduleLock, 23) +PG_LWLOCK(SyncScanLock, 24) +PG_LWLOCK(RelationMappingLock, 25) +/* 26 was NotifySLRULock */ +PG_LWLOCK(NotifyQueueLock, 27) +PG_LWLOCK(SerializableXactHashLock, 28) +PG_LWLOCK(SerializableFinishedListLock, 29) +PG_LWLOCK(SerializablePredicateListLock, 30) +/* 31 was SerialSLRULock */ +PG_LWLOCK(SyncRepLock, 32) +PG_LWLOCK(BackgroundWorkerLock, 33) +PG_LWLOCK(DynamicSharedMemoryControlLock, 34) +PG_LWLOCK(AutoFileLock, 35) +PG_LWLOCK(ReplicationSlotAllocationLock, 36) +PG_LWLOCK(ReplicationSlotControlLock, 37) +/* 38 was CommitTsSLRULock */ +PG_LWLOCK(CommitTsLock, 39) +PG_LWLOCK(ReplicationOriginLock, 40) +PG_LWLOCK(MultiXactTruncationLock, 41) +/* 42 was OldSnapshotTimeMapLock */ +PG_LWLOCK(LogicalRepWorkerLock, 43) +PG_LWLOCK(XactTruncationLock, 44) +/* 45 was XactTruncationLock until removal of BackendRandomLock */ +PG_LWLOCK(WrapLimitsVacuumLock, 46) +PG_LWLOCK(NotifyQueueTailLock, 47) +PG_LWLOCK(WaitEventExtensionLock, 48) +PG_LWLOCK(WALSummarizerLock, 49) +PG_LWLOCK(DSMRegistryLock, 50) +PG_LWLOCK(InjectionPointLock, 51) +PG_LWLOCK(SerialControlLock, 52) -- 2.39.2 --hafsgxjamluc3fye-- ^ permalink raw reply [nested|flat] 15+ messages in thread
end of thread, other threads:[~2024-03-01 12:03 UTC | newest] Thread overview: 15+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-01-12 06:09 Re: Add checkpoint and redo LSN to LogCheckpointEnd log message Julien Rouhaud <[email protected]> 2022-01-12 06:19 ` cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) Tom Lane <[email protected]> 2022-01-12 06:31 ` Re: cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) Julien Rouhaud <[email protected]> 2022-01-12 06:37 ` Re: cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) Tom Lane <[email protected]> 2022-01-12 06:50 ` Re: cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) Julien Rouhaud <[email protected]> 2022-01-12 06:52 ` Re: cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) Thomas Munro <[email protected]> 2022-01-12 06:58 ` Re: cfbot wrangling (was Re: Add checkpoint and redo LSN to LogCheckpointEnd log message) Tom Lane <[email protected]> 2022-01-13 06:20 ` Bharath Rupireddy <[email protected]> 2022-01-14 13:22 ` Bharath Rupireddy <[email protected]> 2022-01-20 00:36 ` Bossart, Nathan <[email protected]> 2022-01-20 03:00 ` Kyotaro Horiguchi <[email protected]> 2022-01-28 03:13 ` Bharath Rupireddy <[email protected]> 2022-01-27 15:07 ` Bharath Rupireddy <[email protected]> 2022-01-27 23:57 ` Nathan Bossart <[email protected]> 2024-03-01 12:03 [PATCH] Rework lwlocknames.txt to become lwlocklist.h Alvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox