Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lbxWn-0007LQ-Ro for pgsql-hackers@arkaria.postgresql.org; Thu, 29 Apr 2021 03:39:49 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lbxWl-0006bG-Bd for pgsql-hackers@arkaria.postgresql.org; Thu, 29 Apr 2021 03:39:47 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lbxWl-0006b9-4Q for pgsql-hackers@lists.postgresql.org; Thu, 29 Apr 2021 03:39:47 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lbxWj-00035q-13 for pgsql-hackers@postgresql.org; Thu, 29 Apr 2021 03:39:46 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 13T3dckR2958652; Wed, 28 Apr 2021 23:39:38 -0400 From: Tom Lane To: Andres Freund cc: Michael Paquier , Thomas Munro , Stephen Frost , Alvaro Herrera , David Steele , Dmitry Dolgov <9erthalion6@gmail.com>, Jakub Wartak , Tomas Vondra , pgsql-hackers Subject: Re: WIP: WAL prefetch (another approach) In-reply-to: <20210429031409.quuhyjihk6hqbloe@alap3.anarazel.de> References: <892344.1619054465@sss.pgh.pa.us> <20210428163245.ff67pb2ii5irr5zr@alap3.anarazel.de> <2808045.1619628335@sss.pgh.pa.us> <2947529.1619652293@sss.pgh.pa.us> <20210429001224.cdkef4ybkw7gramv@alap3.anarazel.de> <2949950.1619655883@sss.pgh.pa.us> <20210429005922.ftonkkpijkhhlazf@alap3.anarazel.de> <20210429031409.quuhyjihk6hqbloe@alap3.anarazel.de> Comments: In-reply-to Andres Freund message dated "Wed, 28 Apr 2021 20:14:09 -0700" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <2958650.1619667578.1@sss.pgh.pa.us> Content-Transfer-Encoding: quoted-printable Date: Wed, 28 Apr 2021 23:39:38 -0400 Message-ID: <2958651.1619667578@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Andres Freund writes: > I was now able to reproduce the problem again, and I'm afraid that the > bug I hit is likely separate from Tom's. Yeah, I think so --- the symptoms seem quite distinct. My score so far today on the G4 is: 12 error-free regression test cycles on b3ee4c503 (plus one more with shared_buffers set to 16MB, on the strength of your previous hunch --- didn't fail for me though) HEAD failed on the second run with the same symptom as before: 2021-04-28 22:57:17.048 EDT [50479] FATAL: inconsistent page found, rel 1= 663/58183/69545, forknum 0, blkno 696 2021-04-28 22:57:17.048 EDT [50479] CONTEXT: WAL redo at 4/B72D408 for He= ap/INSERT: off 77 flags 0x00; blkref #0: rel 1663/58183/69545, blk 696 FPW This seems to me to be pretty strong evidence that I'm seeing *something* real. I'm currently trying to isolate a specific commit to pin it on. A straight "git bisect" isn't going to work because so many people had broken so many different things right around that date :-(, so it may take awhile to get a good answer. regards, tom lane