Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nUIU9-00068d-57 for pgsql-hackers@arkaria.postgresql.org; Wed, 16 Mar 2022 01:29:57 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1nUIU8-0001XP-2Q for pgsql-hackers@arkaria.postgresql.org; Wed, 16 Mar 2022 01:29:56 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nUIU7-0001XG-PV for pgsql-hackers@lists.postgresql.org; Wed, 16 Mar 2022 01:29:55 +0000 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1nUIU4-0001c6-NJ for pgsql-hackers@lists.postgresql.org; Wed, 16 Mar 2022 01:29:54 +0000 Received: by mail-pf1-x42c.google.com with SMTP id l8so1777219pfu.1 for ; Tue, 15 Mar 2022 18:29:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:message-id:to:cc:subject:from:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=TThSX0ZgoCfUTeLl2gXF/Gd9z3l5zbdAmykqoYbISWk=; b=OmcXHg0wm2F1n4SZwsh8qewwk7Phc/a05IbNQIu2etENfv6H+LRQ+hjLLbsGxrEJwi 4oBGlRFYwu8t1ob48EJ1K8/QDflvB3Bz4zgum6yOja8VEw1LD8bdTMQhDAou1lNRcD4b irifIRuyPozEJafWOeEU5N26aJqBGIMGZ/I6+xFQvxgt9nBwzZ9Czu+S0NqE+ZGIdP+9 uqzElH6v1m6smnOQEJF07XWiySEtU6Suz9l0Q95WOXtP/YBRJgmnpXxbGvFPKYDFsDJh 6XaGTPA/GbhdUDqJfy1BTzrHYlUWL5dKMxc+goyJhissclTKtsAO7GWWoCgd/Qt8ebf/ 4jKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:to:cc:subject:from:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=TThSX0ZgoCfUTeLl2gXF/Gd9z3l5zbdAmykqoYbISWk=; b=hYfXLHQYJpOXyPhiXfh+ZuQsDzpd1SMb/5PiOFiteXIq+M5xWcSG0ccqAma9eks5No luRXBw3W6h+pprmtFsYSWR4zXngRi69a9BRetA96rWcPNslCZEnV+qbjTAq6eqBhHTLW ZiEl2AwjZ09Wc3DmrLbGh67WqpmRPmQ3y9ODTVt3lR9qqVdIUomZemCzl25GNN3nmESx rfJe9a3ZE3Y+IIR5gXWy2tQwOsoiITr/dCiaAhEWCuTrHNeoJ1cBKIzKqGMz0php1Abu fu99oRyReBcK5Y7YPu5dHLKV+OOVWRHFU82cOnqrnO+Zsa5dF4LWmIj2YMEDqN+/kOFd vnFw== X-Gm-Message-State: AOAM530Db+TTSpB5Z4BRQvkLkD97o+duWl0Jni/7N5AhUfKCYxyr7hno FeAOxWrY+HJq11lB2Fsv4ko= X-Google-Smtp-Source: ABdhPJxOX7QVTj7eAgeOQsu0SZYGGw6iOk+39wuD3oCoVMTT+VmMQuX0gcDt2miSRsDFV443KcridQ== X-Received: by 2002:a63:e716:0:b0:380:85d1:656c with SMTP id b22-20020a63e716000000b0038085d1656cmr26734956pgi.321.1647394191757; Tue, 15 Mar 2022 18:29:51 -0700 (PDT) Received: from localhost (KD036014041111.ppp-bb.dion.ne.jp. [36.14.41.111]) by smtp.gmail.com with ESMTPSA id b16-20020a056a00115000b004f6ff260c9esm358809pfm.207.2022.03.15.18.29.48 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 15 Mar 2022 18:29:50 -0700 (PDT) Date: Wed, 16 Mar 2022 10:29:47 +0900 (JST) Message-Id: <20220316.102947.378283819592596734.horikyota.ntt@gmail.com> To: michael@paquier.xyz Cc: bharath.rupireddyforpostgres@gmail.com, masao.fujii@oss.nttdata.com, nathandbossart@gmail.com, sfrost@snowman.net, bossartn@amazon.com, rjuju123@gmail.com, pgsql-hackers@lists.postgresql.org Subject: Re: Add checkpoint and redo LSN to LogCheckpointEnd log message From: Kyotaro Horiguchi In-Reply-To: <20220316.091913.806120467943749797.horikyota.ntt@gmail.com> References: <20220315.172340.1059971522574284501.horikyota.ntt@gmail.com> <20220316.091913.806120467943749797.horikyota.ntt@gmail.com> User-Agent: Mew version 6.8 on Emacs 26.1 Mime-Version: 1.0 Content-Type: Multipart/Mixed; boundary="--Next_Part(Wed_Mar_16_10_29_47_2022_770)--" Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk ----Next_Part(Wed_Mar_16_10_29_47_2022_770)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit At Wed, 16 Mar 2022 09:19:13 +0900 (JST), Kyotaro Horiguchi wrote in > In short, I split out the two topics other than checkpoint log to > other threads. So, this is about the main topic of this thread, adding LSNs to checkpint log. Other topics have moved to other treads [1], [2] , [3]. I think this is no longer controversial alone. So this patch is now really Read-for-Commiter and is waiting to be picked up. regards. [1] https://www.postgresql.org/message-id/20220316.102444.2193181487576617583.horikyota.ntt%40gmail.com [2] https://www.postgresql.org/message-id/20220316.102900.2003692961119672246.horikyota.ntt%40gmail.com [3] https://www.postgresql.org/message-id/20220316.102509.785466054344164656.horikyota.ntt%40gmail.com -- Kyotaro Horiguchi NTT Open Source Software Center ----Next_Part(Wed_Mar_16_10_29_47_2022_770)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v13-0001-Add-checkpoint-and-redo-LSN-to-LogCheckpointEnd-.patch" From 4c3d40e7aaf29cb905f3561a291d35981d762456 Mon Sep 17 00:00:00 2001 From: Bharath Rupireddy Date: Fri, 4 Mar 2022 13:22:41 +0900 Subject: [PATCH v13] Add checkpoint and redo LSN to LogCheckpointEnd log message It is useful (for debugging purposes) if the checkpoint end message has the checkpoint LSN(end) and REDO LSN(start). It gives more context while analyzing checkpoint-related issues. The pg_controldata gives the last checkpoint LSN and REDO LSN, but having this info alongside the log message helps analyze issues that happened previously, connect the dots and identify the root cause. --- src/backend/access/transam/xlog.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index ed16f279b1..b85c76d8f8 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -6121,7 +6121,8 @@ LogCheckpointEnd(bool restartpoint) "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " - "distance=%d kB, estimate=%d kB", + "distance=%d kB, estimate=%d kB; " + "lsn=%X/%X, redo lsn=%X/%X", CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, @@ -6134,14 +6135,21 @@ LogCheckpointEnd(bool restartpoint) longest_msecs / 1000, (int) (longest_msecs % 1000), average_msecs / 1000, (int) (average_msecs % 1000), (int) (PrevCheckPointDistance / 1024.0), - (int) (CheckPointDistanceEstimate / 1024.0)))); + (int) (CheckPointDistanceEstimate / 1024.0), + /* + * ControlFileLock is not required as we are the only + * updator of these variables. + */ + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo)))); else ereport(LOG, (errmsg("checkpoint complete: wrote %d buffers (%.1f%%); " "%d WAL file(s) added, %d removed, %d recycled; " "write=%ld.%03d s, sync=%ld.%03d s, total=%ld.%03d s; " "sync files=%d, longest=%ld.%03d s, average=%ld.%03d s; " - "distance=%d kB, estimate=%d kB", + "distance=%d kB, estimate=%d kB; " + "lsn=%X/%X, redo lsn=%X/%X", CheckpointStats.ckpt_bufs_written, (double) CheckpointStats.ckpt_bufs_written * 100 / NBuffers, CheckpointStats.ckpt_segs_added, @@ -6154,7 +6162,13 @@ LogCheckpointEnd(bool restartpoint) longest_msecs / 1000, (int) (longest_msecs % 1000), average_msecs / 1000, (int) (average_msecs % 1000), (int) (PrevCheckPointDistance / 1024.0), - (int) (CheckPointDistanceEstimate / 1024.0)))); + (int) (CheckPointDistanceEstimate / 1024.0), + /* + * ControlFileLock is not required as we are the only + * updator of these variables. + */ + LSN_FORMAT_ARGS(ControlFile->checkPoint), + LSN_FORMAT_ARGS(ControlFile->checkPointCopy.redo)))); } /* -- 2.27.0 ----Next_Part(Wed_Mar_16_10_29_47_2022_770)----