Subject: Re: AIO writes vs hint bits vs checksums

From: Andres Freund <[email-masked]>
Date: Tue, 1 Apr 2025 11:08:34 -0400

Hi,

On 2025-04-01 13:34:53 +0300, Heikki Linnakangas wrote:
> Here's a rebase of these patches.

Thanks!

Curious what made you do this? Do you need any parts of this soon?

I've been hacking on this a bit more, btw, just haven't gotten around to the
necessary README rephrasings... :(

> I went ahead and committed the "heapam: Only set tuple's block once per page
> in pagemode" patch, because it was trivial and independent of the rest.

Thanks!

I think I should just commit the kill_prior_tuples test, it's better than
nothing.

Greetings,

Andres Freund

---

From: Nico Williams <[email-masked]>
Date: Tue, 1 Apr 2025 10:29:31 -0500

On Wed, Oct 30, 2024 at 02:16:51PM +0200, Heikki Linnakangas wrote:
> Acquiring the exclusive lock in step 4 is just a way to wait for all the
> existing share-lockers to release the lock. You wouldn't need to block new
> share-lockers. We have LW_WAIT_UNTIL_FREE, which is almost what we need, but
> it currently ignores share-lockers. So doing this "properly" would require
> more changes to LWLocks. Briefly acquiring an exclusive lock seems
> acceptable though.

The problem is starvation.  For this you really want something more like
rwlocks that do not have the writer starvation problem.  But rwlocks
have other problems too, like in this case forcing readers to wait.

What you want here is something more like a barrier where readers that
did not see that the page has BM_IO_IN_PROGRESS set get to act as though
it's not set while readers that did see that the page has
BM_IO_IN_PROGRESS set don't, and the process that did set that bit gets
to wait for the first set of readers all without blocking the second set
of readers.  That's something akin to an rwlock, but better -- in fact,
it resembles RCU.

Nico
--

---

From: Andres Freund <[email-masked]>
Date: Wed, 13 Aug 2025 15:38:14 -0400

Hi,

On 2024-10-30 12:45:27 -0400, Andres Freund wrote:
> On 2024-10-30 13:29:01 +0200, Heikki Linnakangas wrote:
> > On 30/10/2024 04:21, Andres Freund wrote:
> > > Attached is a, unfortunately long, series of patches implementing what I
> > > described upthread.
> > 
> > Review of the preparatory patches:
> > 
> > > 0001 Add very basic test for kill_prior_tuples
> > > 
> > >      We currently don't exercise this patch for gist and hash, which seems
> > >      somewhat criminal.
> > 
> > Interesting to use the isolationtester for this. There's just one session,
> > so you're just using it to define reusable steps with handy names.
> 
> Yea. I had started out writing it as a pg_regress style test and it quickly got very
> hard to understand.
> 
> 
> > I'm fine with that, but please add a comment to explain it.
> 
> Makes sense.
> 
> 
> > I wonder if it'd be more straightforward to make it a regular pg_regress
> > test though. There would be some repetition, but would it be so bad?
> 
> I found it to be quite bad. If testing just one AM it's ok-ish, but once you
> test 2-3 it gets very long and repetitive. I guess we could use functions or
> such to make it a bit less painful - but that point, is it actually simpler?
> 
> 
> > You forgot to add the new test to 'isolation_schedule'.
> 
> Oops.
> 
> > typos:
> >  "inex" -> "index"
> >  "does something approximately reasonble" -> "do something approximately
> > reasonable"
> 
> Oops^2.

Pushed the test with these changes.

Greetings,

Andres Freund

---

From: Maxim Orlov <[email-masked]>
Date: Mon, 27 Oct 2025 14:48:43 +0300

> Pushed the test with these changes.

I think there is one thing that could be improved here. The isolation
test index-killtuples (377b7ab1452) adds a new dependency:
contrib/btree_gin. contrib/btree_gist. This must be taken into
account in the makefile.

Prior to this patch, isolation tests could be carried out independently
using "make check -C src/test/isolation", but they now require make
check-world to execute successfully, otherwise this test fails with:
====
 step create_ext_btree_gist: CREATE EXTENSION btree_gist;
+ERROR:  extension "btree_gist" is not available
 step create_gist: CREATE INDEX kill_prior_tuple_gist ON kill_prior_tuple
USING gist (key);
+ERROR:  data type integer has no default operator class for access method
"gist"
 step flush: SELECT FROM pg_stat_force_next_flush();
====

-- 
Best regards,
Maxim Orlov.

---

From: Michael Paquier <[email-masked]>
Date: Tue, 28 Oct 2025 13:06:53 +0900

> contrib/btree_gin. contrib/btree_gist. This must be taken into

Also mentioned here:
https://postgr.es/m/[email-masked]

It looks that the conclusion would be to move this test into its own
test module to avoid the external contrib/ dependency in
src/test/isolation/, as duplicated Makefile rules in
src/test/isolation/Makefile are unpleasant.
--
Michael