Message-ID: <c5e114244d391c5acc266ca7abeffe389595e286.camel@j-davis.com>
Subject: Re: Comments on Custom RMGRs
From: Jeff Davis <pgsql@j-davis.com>
To: Simon Riggs <simon.riggs@enterprisedb.com>
Cc: Andres Freund <andres@anarazel.de>, PostgreSQL Hackers
	 <pgsql-hackers@postgresql.org>
Date: Fri, 13 May 2022 08:46:24 -0700
In-Reply-To: 
 <CANbhV-ESya+5rO=TCqdqBn5AqYoLU2Tvq6VwHhQE_OFVSCYzOA@mail.gmail.com>
References: 
	<CANbhV-E4pTWeF-DsdaGsOrjJNFWPaR+Dstjrnkqvf9JFFgOKKQ@mail.gmail.com>
	 <f781f8f198567cee9e3776487bab3e41a7b91b82.camel@j-davis.com>
	 <20220512034010.4oqa76pasrulkw32@alap3.anarazel.de>
	 <CANbhV-G7SfdRLR6r7sxY0uL=TRU0S=cxkWJGWyTdss3u8LFK=g@mail.gmail.com>
	 <c3645cf3e6a30c922c5dec91e908ea70f5758133.camel@j-davis.com>
	 <CANbhV-ESya+5rO=TCqdqBn5AqYoLU2Tvq6VwHhQE_OFVSCYzOA@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/c5e114244d391c5acc266ca7abeffe389595e286.camel%40j-davis.com>
Precedence: bulk

On Fri, 2022-05-13 at 13:31 +0100, Simon Riggs wrote:
> The first was a discussion about a data structure needed by BDR about
> 4 years ago. In the absence of a pluggable checkpoint, the solution
> was forced to use a normal table, which wasn't very satisfactory.

I'm interested to hear more about this case. Are you developing it into
a full table AM? In my experience with columnar, there's still a long
tail of things I wish I had in the backend to better support complete
table AMs.

> The second was a more recent conversation with Mike Stonebraker, at
> the end of 2021.. He was very keen to remove the buffer manager
> entirely, which requires that we have a new smgr, which then needs
> new
> code to allow it to be written to disk at checkpoint time, which then
> requires some kind of pluggable code at checkpoint time. (Mike was
> also keen to remove WAL, but that's another story entirely!).

I'm guessing that would be more of an experimental/ambitious project,
and based on modified postgres anyway.

> The last use case was unlogged indexes, which need to be read from
> disk at startup or rebuilt after crash, which requires RmgrStartup to
> work both with and without InRedo=true.

That sounds like a core feature, in which case we can just refactor
that for v16. It might be a nice cleanup for unlogged tables, too. I
don't think your 002-v2 patch is particularly risky, but any reluctance
at all probably pushes it to v16 given that it's so late in the cycle.

Regards,
	Jeff Davis