Message-ID: <f80dca22db5578eccb8242c7af58061f9a9b9ced.camel@j-davis.com>
Subject: Re: Improve WALRead() to suck data directly from WAL buffers when
 possible
From: Jeff Davis <pgsql@j-davis.com>
To: Andres Freund <andres@anarazel.de>
Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Alvaro
 Herrera <alvherre@alvh.no-ip.org>, Dilip Kumar <dilipbalaut@gmail.com>,
 Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
 pgsql-hackers@lists.postgresql.org, Nathan Bossart
 <nathandbossart@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>
Date: Mon, 12 Feb 2024 17:33:24 -0800
In-Reply-To: <20240212233648.4ephivkh7ukkiv2k@awork3.anarazel.de>
References: 
	<CALj2ACW4oZzmg6joL0ppQN=cCGOLaVKJknc3OLqmo8Q+cGDRKw@mail.gmail.com>
	 <202401301731.o3cthlcry4ve@alvherre.pgsql>
	 <CALj2ACVSR9cgFW=OMxd4mtLceAmucVYHJ2anRAoG3y6OSyLR+g@mail.gmail.com>
	 <db1e59dbdaf0c2fe722f1c5dd534c7cb048ba606.camel@j-davis.com>
	 <20240212201853.r6dl25onohrwagxt@awork3.anarazel.de>
	 <edf041b75c4cc59787f614998b4619b06dddfb06.camel@j-davis.com>
	 <20240212233648.4ephivkh7ukkiv2k@awork3.anarazel.de>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu2 
MIME-Version: 1.0
Archived-At: <https://www.postgresql.org/message-id/f80dca22db5578eccb8242c7af58061f9a9b9ced.camel%40j-davis.com>
Precedence: bulk

On Mon, 2024-02-12 at 15:36 -0800, Andres Freund wrote:
>=20
> It doesn't really seem like a necessary, or even particularly useful,
> part. You couldn't just call WALRead() for that, since the caller
> would need
> to know the range up to which WAL is valid but not yet flushed as
> well. Thus
> the caller would need to first use WaitXLogInsertionsToFinish() or
> something
> like it anyway - and then there's no point in doing the WALRead()
> anymore.

I follow until the last part. Did you mean "and then there's no point
in doing the WaitXLogInsertionsToFinish() in WALReadFromBuffers()
anymore"?

For now, should I assert that the requested WAL data is before the
Flush pointer or assert that it's before the Write pointer?

> Note that for replicating unflushed data, we *still* might need to
> fall back
> to reading WAL data from disk. In which case not asserting in
> WALRead() would
> just make it hard to find bugs, because not using
> WaitXLogInsertionsToFinish()
> would appear to work as long as data is in wal buffers, but as soon
> as we'd
> fall back to on-disk (but unflushed) data, we'd send bogus WAL.

That makes me wonder whether my previous idea[1] might matter: when
some buffers have been evicted, should WALReadFromBuffers() keep going
through the loop and return the end portion of the requested data
rather than the beginning?

We can sort that out when we get closer to replicating unflushed WAL.

Regards,
	Jeff Davis

[1]=20
https://www.postgresql.org/message-id/2b36bf99e762e65db0dafbf8d338756cf5fa6=
ece.camel@j-davis.com