Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nuPGH-000269-W8 for pgsql-hackers@arkaria.postgresql.org; Fri, 27 May 2022 01:59:34 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1nuPGG-0008JN-JZ for pgsql-hackers@arkaria.postgresql.org; Fri, 27 May 2022 01:59:32 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nuPGG-0008J8-84 for pgsql-hackers@lists.postgresql.org; Fri, 27 May 2022 01:59:32 +0000 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1nuPGE-0005a7-Ga for pgsql-hackers@postgresql.org; Fri, 27 May 2022 01:59:31 +0000 Received: by mail-pj1-x1036.google.com with SMTP id m14-20020a17090a414e00b001df77d29587so5925600pjg.2 for ; Thu, 26 May 2022 18:59:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:message-id:to:cc:subject:from:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=3mRMzgEgj8c3Mri42s+4Ou071dEQaxXzFHIhAvyiH28=; b=bMwwR7qM7QVWguUgthNbH4LvpgLT2QSUfxPZoyiKn6RUy/lq9Gisk2g6S0zFu5D0GJ 03AyQ7R24hkTwj6n6p/YaGqrG/ncsLfCvQUPVjKJ+o8NJPDr4G2R5ZgR2qCo9qbtIHnx 6PCoqbtUQDp8pkp0f37N4pXGV39LDTWp+kfvzq0ja2z0UUdzpME/PdTnw670xHr1Pivi WhRhTqv/PH2gVyX67CnkZl7dwFwdc58Hb9KlioduHSczfIH1sys6HsS5TsVul1LTA5me KY9CF0zzWuFnyRAtLhjY93ARuAoyuXlfE2SjmneKyZngDITZbYCm+RIqun/u4IMIAYEr FclA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:to:cc:subject:from:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=3mRMzgEgj8c3Mri42s+4Ou071dEQaxXzFHIhAvyiH28=; b=4n4i9tG25WIm04yN1waociAjKItGTV1oZkVomcbi63dNldjT5l8uIj3y5bteEG3LPo 0M9aOOAURwfLubPiv2HMduxBgixMg+NQXUHdWofIJKeZ8U34IN1yUD2PLiW1HshMLmQu lz2TIWooYzJ4MbcFzZVNFqHbzsXE4UGRBIv3+hamq5D4qCaB3iCnQCr/sxKl4FlVmnZv zEAJVVoiPfZoSckRuNm8H/e3oAA6RYNIXJOFXKUvmL5nI7aleTLBtLdImsgmF2aAOE06 D1oRFRK63yYN14vgNVggax7+livmswcGL/uBzl4/pFUA7U+z/XxUedd/+pmWHN6i5FtW OH0w== X-Gm-Message-State: AOAM5339U3tTyJVqTELJGjFDvb9G+hUiowvVp57wLpuUixp3s7NoUfup J0IsU5wthIcCbWVhrdcQHsc= X-Google-Smtp-Source: ABdhPJyV2q8+EwJLZ1c49DSCibeYtqeGey3tlzv6P+7A1GtC4eKI3xHMgclLFjw5CutBgI9iiEU+4w== X-Received: by 2002:a17:90a:b78d:b0:1d9:4f4f:bc2a with SMTP id m13-20020a17090ab78d00b001d94f4fbc2amr5701420pjr.155.1653616768639; Thu, 26 May 2022 18:59:28 -0700 (PDT) Received: from localhost (KD036014041111.ppp-bb.dion.ne.jp. [36.14.41.111]) by smtp.gmail.com with ESMTPSA id q6-20020a170902f78600b001635c9e7f77sm2290604pln.57.2022.05.26.18.59.26 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 26 May 2022 18:59:27 -0700 (PDT) Date: Fri, 27 May 2022 10:59:24 +0900 (JST) Message-Id: <20220527.105924.1864299458038443589.horikyota.ntt@gmail.com> To: simseih@amazon.com Cc: alvherre@alvh.no-ip.org, pgsql-hackers@postgresql.org Subject: Re: [BUG] Panic due to incorrect missingContrecPtr after promotion From: Kyotaro Horiguchi In-Reply-To: References: User-Agent: Mew version 6.8 on Emacs 26.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk At Thu, 26 May 2022 19:57:41 +0000, "Imseih (AWS), Sami" wrote in > We see another occurrence of this bug with the last patch applied in 13.7. > > After a promotion we observe the following in the logs: ... > We think it's because VerifyOverwriteContrecord was not > called which is why we see this behavior. > > Are there are other places where missingContrecPtr > should be invalidated, such as after a successful promotion? The only cause known to me for EndOfLog being moved to such location is missingContrecPtr. But if the next record is not XLOG_OVERWRITE_CONTRECORD recovery should have stopped there. And if XLOG_OVERWRITE_CONTRECORD is there, VerifyOverwriteContrecord should have been called.. Could you inspect WAL files of the environment and see if the first record of the '7/A8'th segment OVERWRITE_CONTRECORD? I don't say that makes some progress on this, but could be the first step. regards. -- Kyotaro Horiguchi NTT Open Source Software Center