Message-ID: <73df9831747b01ced949102610dc44ca6026ee1c.camel@j-davis.com>
Subject: Re: Improve WALRead() to suck data directly from WAL buffers when
 possible
From: Jeff Davis <pgsql@j-davis.com>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Dilip Kumar <dilipbalaut@gmail.com>,
  Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
 pgsql-hackers@lists.postgresql.org, Nathan Bossart
 <nathandbossart@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>
Date: Thu, 04 Jan 2024 17:50:05 -0800
In-Reply-To: <CALj2ACXy5eA-Y4Qgm3MUYZTGr_opNm2SzGtN+Zj6dbcSzqHrJA@mail.gmail.com>
References: <0d84c025e38673cb575dcf4987d050fe7b1e1a1f.camel@j-davis.com>
	 <20231011224353.cl7c2s222dw3de4j@awork3.anarazel.de>
	 <CALj2ACU2DGH-BB_187N7gfVkLdxfJ+doVTcteS0ugLn_c8mLOw@mail.gmail.com>
	 <CALj2ACVgVgA5BSSrEYO2eTMEGB=QUbcYosYm3vZ3R2=GPB6tNw@mail.gmail.com>
	 <5c6e8e60c5c156a9f51ccf850f70829e45a985a1.camel@j-davis.com>
	 <CALj2ACVjzHgQvONnTLffyBJRACPwhsf-cYG4TQ2KHKrFfFop-w@mail.gmail.com>
	 <4132fe48f831ed6f73a9eb191af5fe475384969c.camel@j-davis.com>
	 <CALj2ACV123Ke6wn05QCnCgJhFpcarGCv7sKM1p39t+qpUNHUgQ@mail.gmail.com>
	 <20231104004348.zhxzk2p266i3r2z2@awork3.anarazel.de>
	 <CALj2ACUSydTHPcucftm4XdiJ4osH8zeNENyADd-zfDzEMf6akA@mail.gmail.com>
	 <20231109205836.zjoawdrn4q77yemv@awork3.anarazel.de>
	 <CALj2ACXo-_boM3h-PJuvfuvFzP1nv2wcZBoZN_YbRbQBGQYAUg@mail.gmail.com>
	 <CALj2ACVfFMfqD5oLzZSQQZWfXiJqd-NdX0_317veP6FuB31QWA@mail.gmail.com>
	 <3750ff9c2a5556808827e7fc5bcc0bd16e9facf4.camel@j-davis.com>
	 <CALj2ACXy5eA-Y4Qgm3MUYZTGr_opNm2SzGtN+Zj6dbcSzqHrJA@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu2 
MIME-Version: 1.0
Archived-At: <https://www.postgresql.org/message-id/73df9831747b01ced949102610dc44ca6026ee1c.camel%40j-davis.com>
Precedence: bulk

On Wed, 2023-12-20 at 15:36 +0530, Bharath Rupireddy wrote:
> Thanks. Attaching remaining patches as v18 patch-set after commits
> c3a8e2a7cb16 and 766571be1659.

Comments:

I still think the right thing for this patch is to call
XLogReadFromBuffers() directly from the callers who need it, and not
change WALRead(). I am open to changing this later, but for now that
makes sense to me so that we can clearly identify which callers benefit
and why. I have brought this up a few times before[1][2], so there must
be some reason that I don't understand -- can you explain it?

The XLogReadFromBuffersResult is never used. I can see how it might be
useful for testing or asserts, but it's not used even in the test
module. I don't think we should clutter the API with that kind of thing
-- let's just return the nread.

I also do not like the terminology "partial hit" to be used in this
way. Perhaps "short read" or something about hitting the end of
readable WAL would be better?

I like how the callers of WALRead() are being more precise about the
bytes they are requesting.

You've added several spinlock acquisitions to the loop. Two explicitly,
and one implicitly in WaitXLogInsertionsToFinish(). These may allow you
to read slightly further, but introduce performance risk. Was this
discussed?

The callers are not checking for XLREADBUGS_UNINITIALIZED_WAL, so it
seems like there's a risk of getting partially-written data? And it's
not clear to me the check of the wal page headers is the right one
anyway.

It seems like all of this would be simpler if you checked first how far
you can safely read data, and then just loop and read that far. I'm not
sure that it's worth it to try to mix the validity checks with the
reading of the data.

Regards,
	Jeff Davis

[1]=C2=A0https://www.postgresql.org/message-id/4132fe48f831ed6f73a9eb191af5=
fe475384969c.camel%40j-davis.com
[2]
https://www.postgresql.org/message-id/2ef04861c0f77e7ae78b703770cc2bbbac3d8=
5e6.camel@j-davis.com