Subject: Re: [HACKERS] Logical replication and multimaster

From: Jon Erdman <[email-masked]>
Date: Thu, 5 Oct 2023 05:31:00 +0000

Oops,

Looking at my old message from 2015 on this subject, it was Magnus who 
pish-poshed it, and Page who liked it. Don't want to throw shaded at the 
wrong person ;)
--
Jon Erdman (aka StuckMojo)
     PostgreSQL Zealot

On 10/5/23 12:14 AM, Jon Erdman wrote:
> 
> Well,
> 
> Given my earlier message about Apple wanting to pay me for writing 
> patches now, maybe I can revisit this idea.
> 
> For background: I brought up the idea of an FDW that could read custom 
> dump files and expose them as tables so that you could grab just a 
> single record (or of course more) from a custom dump file without having 
> to load the whole thing up, if you were stuck reaching into a backup to 
> get at accidentally deleted tables, rows, etc.. The stopper, which was 
> communicated to me by Tom at the following pgcon was that the code for 
> parsing custom dumps is duplicated in pg_dump only, and would need to be 
> duplicated into the server for the FDW, or broken out into a library.
> 
> And for posterity, Dave Page said that was a stupid idea, while Magnus 
> said that it sounded useful. And IIRC Bruce and Robert H said it was 
> doable, just a good deal of work on the refactor needed.
> 
> This convo went down at the Amsterdam conf where I spoke about using 
> writeable LVM snapshots to expose each developer a copy of prod to 
> noodle on, without having to actually make a full copy for each dev.
> 
> Added trivia: I gave the talk with a can of Heineken in my hand at the 
> podium, and my lightning talk had the work F(&king Cool in the title ;)
> 
> That was also when I bought my plush Slony which was supposedly the very 
> last one. (turns out they made more)
> -- 
> Jon Erdman (aka StuckMojo)
>      PostgreSQL Zealot
> 
> On 12/15/15 9:48 PM, Jim Nasby wrote:
>> On 12/13/15 7:37 AM, David Fetter wrote:
>>> As I understand it, pushing these into a library has been proposed but
>>> not rejected.  That it hasn't happened yet is mostly about the lack of
>>> tuits (the round ones) to rewrite the functionality as libraries and
>>> refactor pg_dump/pg_restore to use only calls to same.  As usual, it's
>>> less about writing the code and more about the enormous amount of
>>> testing any such a refactor would entail.
>>
>> My understanding as well. IIRC Jon Erdman brought this question up a 
>> couple years ago and the response was "It'd probably be accepted, it's 
>> just that no one has done the work."
>>
>>> I believe that refactoring much of pg_dump's functionality for the
>>> current version of the server into SQL-accessible functions and making
>>> pg_dump use only those functions is achievable with available
>>> resources.
>>>
>>> Such a refactor need not be all-or-nothing.  For example, the
>>> dependency resolution stuff is a first step that appears to be worth
>>> doing by itself even if the effort then pauses, possibly for some
>>> time.
>>
>> If someone wanted to spend time on this, I suspect it'd be worth 
>> looking at how bad some of the backward compatibility issues would be 
>> if done in the server. Maybe they wouldn't be that bad. I suspect the 
>> audience for this code would be much larger if it was in the server as 
>> opposed to a C library.