Subject: Re: WIP: WAL prefetch (another approach)
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Stephen Frost <sfrost@snowman.net>, Andres Freund <andres@anarazel.de>,
 Jakub Wartak <Jakub.Wartak@tomtom.com>,
 Alvaro Herrera <alvherre@2ndquadrant.com>,
 Tomas Vondra <tomas.vondra@2ndquadrant.com>,
 Dmitry Dolgov <9erthalion6@gmail.com>, David Steele <david@pgmasters.net>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
References: 
 <CA+hUKGJ1=pOiNjSgXYJnjE3OyRtp8tjMRcON256e5EFpzPpgtA@mail.gmail.com>
 <VI1PR0701MB69608CBCE44D80857E59572EF6CA0@VI1PR0701MB6960.eurprd07.prod.outlook.com>
 <CA+hUKGKFeYPL9K+SRixcsx1+6HsHhqK+POZyrnnZjw1jERpGcQ@mail.gmail.com>
 <20210210215033.GA27507@tamriel.snowman.net>
 <103ec76d-8bbc-09e1-460c-8785751431b5@enterprisedb.com>
 <20210212044614.5wqjkjbcmbwzever@alap3.anarazel.de>
 <20210213213930.GM27507@tamriel.snowman.net>
 <eed83eaa-91e1-91fe-32d0-ece36c0fa810@enterprisedb.com>
 <20210214231815.GO27507@tamriel.snowman.net>
 <17b3f450-048c-f3a6-7086-259523f33cbc@enterprisedb.com>
 <20210317214331.GF20766@tamriel.snowman.net>
 <28c3352e-5388-6410-aae7-05907079e6c9@enterprisedb.com>
 <CA+hUKGLHTjbBPFX6E1ZCGVyQD30-vWtH=N6EcHEzW13q+jh7FQ@mail.gmail.com>
 <8dd2dcc2-0b6b-a4ab-c1b3-032c0a5ab1c3@enterprisedb.com>
 <CA+hUKGKmkKfH1h9XG5ctYCgqeOHGdN5bHMoQMZLR=+kLY0PQzA@mail.gmail.com>
 <CA+hUKGLsF8sEE+n3qoaV6+XjmYEwYoFVDbLPbYTGz7oGkL=iZA@mail.gmail.com>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
Message-ID: <bab127df-c5c0-6e33-69b1-31471d61ff62@enterprisedb.com>
Date: Wed, 7 Apr 2021 17:27:43 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.8.1
MIME-Version: 1.0
In-Reply-To: 
 <CA+hUKGLsF8sEE+n3qoaV6+XjmYEwYoFVDbLPbYTGz7oGkL=iZA@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/bab127df-c5c0-6e33-69b1-31471d61ff62%40enterprisedb.com>
Precedence: bulk

On 4/7/21 1:24 PM, Thomas Munro wrote:
> Here's rebase, on top of Horiguchi-san's v19 patch set.  My patches
> start at 0007.  Previously, there was a "nowait" flag that was passed
> into all the callbacks so that XLogReader could wait for new WAL in
> some cases but not others.  This new version uses the proposed
> XLREAD_NEED_DATA protocol, and the caller deals with waiting for data
> to arrive when appropriate.  This seems tidier to me.
> 

OK, seems reasonable.

> I made one other simplifying change: previously, the prefetch module
> would read the WAL up to the "written" LSN (so, allowing itself to
> read data that had been written but not yet flushed to disk by the
> walreceiver), though it still waited until a record's LSN was
> "flushed" before replaying.  That allowed prefetching to happen
> concurrently with the WAL flush, which was nice, but it felt a little
> too "special".  I decided to remove that part for now, and I plan to
> look into making standbys work more like primary servers, using WAL
> buffers, the WAL writer and optionally the standard log-before-data
> rule.
> 

Not sure, but the removal seems unnecessary. I'm worried that this will
significantly reduce the amount of data that we'll be able to prefetch.
How likely it is that we have data that is written but not flushed?
Let's assume the replica is lagging and network bandwidth is not the
bottleneck - how likely is this "has to be flushed" a limit for the
prefetching?

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company