Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1njuaJ-0004yu-F6 for pgsql-hackers@arkaria.postgresql.org; Thu, 28 Apr 2022 03:12:51 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1njuZI-0003s9-VH for pgsql-hackers@arkaria.postgresql.org; Thu, 28 Apr 2022 03:11:48 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1njuZI-0003qw-IR for pgsql-hackers@lists.postgresql.org; Thu, 28 Apr 2022 03:11:48 +0000 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1njuZC-0004q8-Pv for pgsql-hackers@lists.postgresql.org; Thu, 28 Apr 2022 03:11:47 +0000 Received: by mail-pj1-x1034.google.com with SMTP id bd19-20020a17090b0b9300b001d98af6dcd1so6520742pjb.4 for ; Wed, 27 Apr 2022 20:11:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=j-davis-com.20210112.gappssmtp.com; s=20210112; h=message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=+YkEGkJq5ntEnjQ1UdULsPnpgsm7F4oS8pMsmJnzcuo=; b=6DFuI3epyU6EdI1in54gg77IdO6ONiFwRe69f2rL5zLHtiyG4TOxuk2Mq1i9ZnLPD1 BLMNav2AS6aGcBBfkIX4S6OL2HKcdAdS/iebHGxJ/sbeYkIuCfxWiK/NWh6/nMNeed5Q bbLRNHEoWQXM5E0tr894A+uG6wi+PLcUnHczVxWuliHlXVut/4C5Mu96aXy+V3r/uJbd Dh+mvBzkrn8nzZ6DUdBJw8Elvl2Y+02lH7WkOPtAUyNSe9ZNITo6vEH9hJZbvbQqAJmX KCJtHgXfsM/6hFujC/hV6MoeCE8OiUikwjITcjBN8eNtGyj7LEWI8hVXUPHMAUCB9Y3v Sldw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=+YkEGkJq5ntEnjQ1UdULsPnpgsm7F4oS8pMsmJnzcuo=; b=VKlK+c5fiRAatt1zfr7dz24Hbtrf4aSZ+2yzt9xImEqt2PpanBs9jpEfk6rVHuA/79 h1IK6GtKKrCmYu/yAVs51m0+GFNakgJJ4jkeP+db1kC1m7Taf/90V518Iwnk5t//lDpF 4tqIRhiYyUIqd211kIv2MVcQH9YYJCollCCxFpp2+cOj/QqO/w4r+fkNNhdqMsyfCTCb BNAFZSrDCBxFm8fKAoir7NNQuEY1RRXuSiGvrbhE+l/hICCovzm4YyXEu1NWNWMD4I9f xxOyHVF0Vg/W8GrFJ41FedAJxWhgA/lUBqgld1zDlysgviYmPBKA6byJ/ZWDHVRyAIgQ zKXA== X-Gm-Message-State: AOAM5313BwbRTvx38YKtA2Hg1xijiYyB4gkxlbflT7dZpFI9Qs/DnsvE wgc17OzxaNnUS3dp6d2TsAEgMA== X-Google-Smtp-Source: ABdhPJzZS7em6N6uk5+j9Q3ZY6x+efHIuEvRtMbOpXJidzirppQH53jm3TTeT9YfEBYtx6lH31MLeQ== X-Received: by 2002:a17:902:e545:b0:15e:5bf3:e4da with SMTP id n5-20020a170902e54500b0015e5bf3e4damr2292290plf.138.1651115500593; Wed, 27 Apr 2022 20:11:40 -0700 (PDT) Received: from jdavis.lan (c-73-231-146-4.hsd1.ca.comcast.net. [73.231.146.4]) by smtp.gmail.com with ESMTPSA id 192-20020a6301c9000000b003c14af50619sm782914pgb.49.2022.04.27.20.11.38 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 27 Apr 2022 20:11:39 -0700 (PDT) Message-ID: Subject: Re: pgsql: Add contrib/pg_walinspect. From: Jeff Davis To: Thomas Munro , Bharath Rupireddy Cc: Tom Lane , Michael Paquier , Noah Misch , Jeff Davis , Postgres hackers Date: Wed, 27 Apr 2022 20:11:37 -0700 In-Reply-To: References: <171769.1650950714@sss.pgh.pa.us> <287795.1651019129@sss.pgh.pa.us> <295868.1651024073@sss.pgh.pa.us> <298689.1651025716@sss.pgh.pa.us> <304865.1651029340@sss.pgh.pa.us> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5-0ubuntu0.18.04.2 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Thu, 2022-04-28 at 12:11 +1200, Thomas Munro wrote: > > Another option might be to abandon this whole no-wait concept and > revert 2258e76f's changes to xlogutils.c. pg_walinspect already does > preliminary checks that LSNs are in range, so you can't enter a value > that will wait indefinitely, and it's interruptible (it's a 1ms > sleep/check loop, not my favourite programming pattern but that's > pre-existing code). If you're unlucky enough to hit the case where > the LSN is judged to be valid but is in the middle of a record that > hasn't been totally flushed yet, it'll just be a bit slower to return > as we wait for the inevitable later flush(es) to happen. The rest of > your record will *surely* be flushed pretty soon (or the flushing > backend panics the whole system and time ends). I don't imagine this > is performance critical work, so maybe that'd be acceptable? I'm inclined toward this option. I was a bit iffy on those xlogutils.c changes to begin with, and they are causing a problem now, so I'd like to avoid layering on more workarounds. The time when we need to wait is very narrow, only in this case where it's earlier than the flush pointer, and the flush pointer is in the middle of a record that's not fully flushed. And as you say, we won't be waiting very long in that case, because once we start to write a WAL record it better finish soon. Bharath, thoughts? When you originally introduced the nowait behavior, I believe that was to solve the case where someone specifies an LSN range well in the future, but we can still catch that and throw an error if we see that it's beyond the flush pointer. Do you see a problem with just doing that and getting rid of the nowait changes? Regards, Jeff Davis