Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p4Z9Q-0004eY-SK for pgsql-hackers@arkaria.postgresql.org; Mon, 12 Dec 2022 03:06:44 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1p4Z9P-00068A-H3 for pgsql-hackers@arkaria.postgresql.org; Mon, 12 Dec 2022 03:06:43 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p4Z9P-000680-8P for pgsql-hackers@lists.postgresql.org; Mon, 12 Dec 2022 03:06:43 +0000 Received: from mail-pj1-x102d.google.com ([2607:f8b0:4864:20::102d]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p4Z9M-000841-Dr for pgsql-hackers@lists.postgresql.org; Mon, 12 Dec 2022 03:06:42 +0000 Received: by mail-pj1-x102d.google.com with SMTP id e7-20020a17090a77c700b00216928a3917so14136527pjs.4 for ; Sun, 11 Dec 2022 19:06:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:from:subject:cc:to:message-id:date:from:to:cc:subject :date:message-id:reply-to; bh=w+T7sxSJ43BA01EpaFl24/T6zegqtK+fZ7s6/6VMkzU=; b=kFE1vSpAYR7u0vczVPKtOY7uSwgesm5fAjPsugxgDLG6u3lFtmfXzuevONm5JOrK+a 0JuhUnez47YUWN5D6ge2thqFcPJ9eBSlX/8M2iTCcYzdaRJwzahOoRwHYJtIly/mJ73s rIF3DK+h6NuON5BQEjnT19lj9X6jdX2NxEGVKANzy5BY1p0iyTVSZXL0uRTnHF8cUsox 1U6NhtPcsZYAdCJQkglcKV1WgUxmqPL3htseBbaIatwMqgr7H5ZrN3AOPvtLBwmIzt4c 6A1L82ZiKNQCt+GukkEKlKiJOgn6WaWYLVZmss4CIQ6pS6YhIEQDwhG+mHfwzFrj1LpD 3rhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:from:subject:cc:to:message-id:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=w+T7sxSJ43BA01EpaFl24/T6zegqtK+fZ7s6/6VMkzU=; b=vFrhtRfb+8TDmUHS850YKIkMDiuQ0eyruO9/qoAJyEhCCnNMTsot6W2nlj69fOsMo5 Q144QYQ8aRAPUePHlMEFfJY0ORZbLT97YjvLV0hHTPXP4XzoowzionsHibiIdEsCyW0r ddZ3cBR+Yd6HaQYWlnQDVj0WUCRS2yNCXYtCBbxQI7ahAmSFOuBe+mCd1MHlw9BwXBdp I1OdEBvcVuUKgc3SUU7Ks3wMjleC5+RSroIw4suqZlRuRz5xItKEdE2JU9qJNK1l2vsI YmeyYqsnyBeuI6+5v5/jvTsP6UvAe8q6N61GL6aCvvEMvLu4BeMZr+R661LQvPrRKUjj meXQ== X-Gm-Message-State: ANoB5pk6kmjDviLNomb4fpGqgvCHAELBZtTPO0atCZ3rnIrVWRQYDEJu NBctQoq9/pmpn6zlmFGtz+qTSUL9xdM= X-Google-Smtp-Source: AA0mqf54+d0yxs3W5Br/hRwOTEU3PxkC7zRthKnDjLONqjuDYvrMzhAotAHcrFmq7+znaVBXwItsgg== X-Received: by 2002:a17:902:6bc8:b0:188:d2b3:26c1 with SMTP id m8-20020a1709026bc800b00188d2b326c1mr13802878plt.10.1670814399510; Sun, 11 Dec 2022 19:06:39 -0800 (PST) Received: from localhost (KD036014041111.ppp-bb.dion.ne.jp. [36.14.41.111]) by smtp.gmail.com with ESMTPSA id h12-20020a170902f54c00b00186ac812ab0sm5042118plf.83.2022.12.11.19.06.37 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 11 Dec 2022 19:06:38 -0800 (PST) Date: Mon, 12 Dec 2022 12:06:36 +0900 (JST) Message-Id: <20221212.120636.567365638124623176.horikyota.ntt@gmail.com> To: bharath.rupireddyforpostgres@gmail.com Cc: pgsql-hackers@lists.postgresql.org Subject: Re: Improve WALRead() to suck data directly from WAL buffers when possible From: Kyotaro Horiguchi In-Reply-To: <20221212.115717.1183698540772933396.horikyota.ntt@gmail.com> References: <20221212.115717.1183698540772933396.horikyota.ntt@gmail.com> User-Agent: Mew version 6.8 on Emacs 26.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk At Mon, 12 Dec 2022 11:57:17 +0900 (JST), Kyotaro Horiguchi wrote in > This patch copies the bleeding edge WAL page without recording the > (next) insertion point nor checking whether all in-progress insertion > behind the target LSN have finished. Thus the copied page may have > holes. That being said, the sequential-reading nature and the fact > that WAL buffers are zero-initialized may make it work for recovery, > but I don't think this also works for replication. Mmm. I'm a bit dim. Recovery doesn't read concurrently-written records. Please forget about recovery. > I remember that the one of the advantage of reading the on-memory WAL > records is that that allows walsender to presend the unwritten > records. So perhaps we should manage how far the buffer is filled with > valid content (or how far we can presend) in this feature. -- Kyotaro Horiguchi NTT Open Source Software Center