Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iZItn-000717-VZ for pgsql-hackers@arkaria.postgresql.org; Mon, 25 Nov 2019 18:15:48 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1iZItl-0005oh-Dj for pgsql-hackers@arkaria.postgresql.org; Mon, 25 Nov 2019 18:15:45 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iZItl-0005oY-5G for pgsql-hackers@lists.postgresql.org; Mon, 25 Nov 2019 18:15:45 +0000 Received: from mail-qt1-x841.google.com ([2607:f8b0:4864:20::841]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iZIth-0004V5-No for pgsql-hackers@postgresql.org; Mon, 25 Nov 2019 18:15:44 +0000 Received: by mail-qt1-x841.google.com with SMTP id g24so11400034qtq.11 for ; Mon, 25 Nov 2019 10:15:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=cJL2uc+EP8jC8N4X86r6OGM61wQfg4QCnMdjTdEIYh8=; b=mJcs6I9lIVijtGp23nC7181k/fPN6q0L0R6l7CfdtrOdUnCc1eLDHtOZm9SgpdhuyX jmUdiwxaDbrOVcRAqNO6u7P1leK88mhUek9orZX1MnmMmNxRn4yyWG7GPUhpq+bRU9cd SPXmZXjMxkYF7c9MmEcMjX5XPcY6QwWMlzwkLftFmcwDf/0eukyChKPOykvhK0VJl0JM BcMCpP8nAgFd3xpSug/wdkZreYWilmv/cNiPPd/eX3/Raidw4wrSx/4kWIszd0esFqBo hz3Z95QFtEtqZw4aIJHpDHY4FXOLYR60gt+mVyHWmqtLnAFt4k5zwAObkXyrxzXybWW8 KxKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=cJL2uc+EP8jC8N4X86r6OGM61wQfg4QCnMdjTdEIYh8=; b=dm5a5ljV0bPLvcxKO0sJ8NNby+EGd9+UZis80ZgVb+2wQfW8ZudhPxYRcDVknY88M/ hT/7SEOmvxr9cLstrB0gIcSBayQ16uPOZ/ej2mIRYjqrGN4yV25OuTy4DgV5X6UAIKwK Hjwty6Yl1Kk4DEZq5T0S2thcBvMQ9BoVoRCJi9J+WuSOsbmoRykJRQIUt2SK3cAMwiIB mfyDjxU3+ielfmR2IO1hjxu015KycZ7Fok7CrCd7NmVce7EFQcfwBD7ie9xjU9ZXlgRf RVIW4JTygRUFlz0rysAlDYC1F14O6Lf5xiufusLxXD8b3nLm1ypyUlKqHnMfYhrePxlk mEwg== X-Gm-Message-State: APjAAAW6WrUDcmxkQhTt1IpCJspLSWVWPrqkgeIBPuXwWoQrjwPTJHPZ WNpPhXExm4Ce5BYHDUAM5kVDfA== X-Google-Smtp-Source: APXvYqzAy/W+XwhBWE/hjrjiai3ixChyY1YIpSxfVtHUg/wJzuIMYEN23zuKy6BzNpPg+Ni/ypKpCg== X-Received: by 2002:aed:33c2:: with SMTP id v60mr17964825qtd.168.1574705739586; Mon, 25 Nov 2019 10:15:39 -0800 (PST) Received: from nimloth.alvh.no-ip.org ([70.32.0.143]) by smtp.gmail.com with ESMTPSA id m22sm3715049qka.28.2019.11.25.10.15.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Nov 2019 10:15:38 -0800 (PST) Received: by nimloth.alvh.no-ip.org (Postfix, from userid 1000) id 7BCBB300B4D; Mon, 25 Nov 2019 15:15:34 -0300 (-03) Date: Mon, 25 Nov 2019 15:15:34 -0300 From: Alvaro Herrera To: Antonin Houska Cc: Michael Paquier , Thomas Munro , Robert Haas , "pgsql-hackers@postgresql.org" Subject: Re: Attempt to consolidate reading of XLOG page Message-ID: <20191125181534.GA19210@alvherre.pgsql> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <50470.1574672520@antos> User-Agent: Mutt/1.10.1 (2018-07-13) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 2019-Nov-25, Antonin Houska wrote: > Alvaro Herrera wrote: > > I see no reason to leave ws_off. We can move that to XLogReaderState; I > > did that here. We also need the offset in WALReadError, though, so I > > added it there too. Conceptually it seems clearer to me this way. > > > > What do you think of the attached? > > It looks good to me. Attached is just a fix of a minor problem in error > reporting that Michael pointed out earlier. Excellent, I pushed it with this change included and some other cosmetic changes. Now there's only XLogPageRead() ... > > BTW I'm not clear what errors can pread()/pg_pread() report that do not > > set errno. I think lines 1083/1084 of WALRead are spurious now. > > All I can say is that the existing calls of pg_pread() do not clear errno, so > you may be right. Right ... in this interface, we only report an error if pg_pread() returns negative, which is documented to always set errno. > I'd appreciate more background about the "partial read" that > Michael mentions here: > > https://www.postgresql.org/message-id/20191125033048.GG37821%40paquier.xyz In the current implementation, if pg_pread() does a partial read, we just loop one more time. I considered changing the "if (readbytes <= 0)" with "if (readbytes < segbytes)", but that seemed pointless. However, writing this now makes me think that we should add a CHECK_FOR_INTERRUPTS in this loop. (I also wonder if we shouldn't limit the number of times we retry if pg_pread returns zero (i.e. no error, but no bytes read either). I don't know if this is a real-world consideration.) -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services