Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lZFkO-0008VX-8n for pgsql-hackers@arkaria.postgresql.org; Wed, 21 Apr 2021 16:30:40 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lZFkN-00026r-6E for pgsql-hackers@arkaria.postgresql.org; Wed, 21 Apr 2021 16:30:39 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lZFkM-00026j-Va for pgsql-hackers@lists.postgresql.org; Wed, 21 Apr 2021 16:30:38 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lZFkI-000898-6V for pgsql-hackers@postgresql.org; Wed, 21 Apr 2021 16:30:37 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 13LGUMNO773933; Wed, 21 Apr 2021 12:30:22 -0400 From: Tom Lane To: Thomas Munro cc: Tomas Vondra , Stephen Frost , Andres Freund , Jakub Wartak , Alvaro Herrera , Tomas Vondra , Dmitry Dolgov <9erthalion6@gmail.com>, David Steele , pgsql-hackers Subject: Re: WIP: WAL prefetch (another approach) In-reply-to: References: <20210210215033.GA27507@tamriel.snowman.net> <103ec76d-8bbc-09e1-460c-8785751431b5@enterprisedb.com> <20210212044614.5wqjkjbcmbwzever@alap3.anarazel.de> <20210213213930.GM27507@tamriel.snowman.net> <20210214231815.GO27507@tamriel.snowman.net> <17b3f450-048c-f3a6-7086-259523f33cbc@enterprisedb.com> <20210317214331.GF20766@tamriel.snowman.net> <28c3352e-5388-6410-aae7-05907079e6c9@enterprisedb.com> <8dd2dcc2-0b6b-a4ab-c1b3-032c0a5ab1c3@enterprisedb.com> Comments: In-reply-to Thomas Munro message dated "Thu, 08 Apr 2021 23:46:21 +1200" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <773931.1619022622.1@sss.pgh.pa.us> Content-Transfer-Encoding: quoted-printable Date: Wed, 21 Apr 2021 12:30:22 -0400 Message-ID: <773932.1619022622@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Thomas Munro writes: > Yeah, it would have been nice to include that but it'll have to be for > v15 due to lack of time to convince myself that it was correct. I do > intend to look into more concurrency of that kind for v15. I have > pushed these patches, updated to be disabled by default. I have a fairly bad feeling about these patches. I've already fixed one critical bug (see 9e4114822), but I am still seeing random, hard to reproduce failures in WAL replay testing. It looks like sometimes the "decoded" version of a WAL record doesn't match what I see in the on-disk data, which I'm having no luck tracing down. Another interesting failure I just came across is 2021-04-21 11:32:14.280 EDT [14606] LOG: incorrect resource manager data = checksum in record at F/438000A4 TRAP: FailedAssertion("state->decoding", File: "xlogreader.c", Line: 845, = PID: 14606) 2021-04-21 11:38:23.066 EDT [14603] LOG: startup process (PID 14606) was = terminated by signal 6: Abort trap with stack trace #0 0x90b669f0 in kill () #1 0x90c01bfc in abort () #2 0x0057a6a0 in ExceptionalCondition (conditionName=3D, errorType=3D, fileName=3D, lineNumber=3D) at assert.c:69 #3 0x000f5cf4 in XLogDecodeOneRecord (state=3D0x1000640, allow_oversized=3D= 1 '\001') at xlogreader.c:845 #4 0x000f682c in XLogNextRecord (state=3D0x1000640, record=3D0xbfffba38, = errormsg=3D0xbfffba9c) at xlogreader.c:466 #5 0x000f695c in XLogReadRecord (state=3D, record=3D0xbfffba98, errormsg=3D) at xlogreader.c:352 #6 0x000e61a0 in ReadRecord (xlogreader=3D0x1000640, emode=3D15, fetching= _ckpt=3D0 '\0') at xlog.c:4398 #7 0x000ea320 in StartupXLOG () at xlog.c:7567 #8 0x00362218 in StartupProcessMain () at startup.c:244 #9 0x000fc170 in AuxiliaryProcessMain (argc=3D, argv=3D) at bootstrap.c:447 #10 0x0035c740 in StartChildProcess (type=3DStartupProcess) at postmaster.= c:5439 #11 0x00360f4c in PostmasterMain (argc=3D5, argv=3D0xa006a0) at postmaster= .c:1406 #12 0x0029737c in main (argc=3D, argv=3D) = at main.c:209 I am not sure whether the checksum failure itself is real or a variant of the seeming bad-reconstruction problem, but what I'm on about right at this moment is that the error handling logic for this case seems quite broken. Why is a checksum failure only worthy of a LOG message? Why is ValidXLogRecord() issuing a log message for itself, rather than being tied into the report_invalid_record() mechanism? Why are we evidently still trying to decode records afterwards? In general, I'm not too pleased with the apparent attitude in this thread that it's okay to push a patch that only mostly works on the last day of the dev cycle and plan to stabilize it later. regards, tom lane