Date: Wed, 9 Feb 2022 19:11:27 +0530
From: Abhijit Menon-Sen <ams@toroid.org>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>,
	Jeevan Ladhe <jeevan.ladhe@enterprisedb.com>,
	Mark Dilger <mark.dilger@enterprisedb.com>,
	pgsql-hackers@postgresql.org,
	tushar <tushar.ahuja@enterprisedb.com>
Subject: Re: refactoring basebackup.c
Message-ID: <YgPEhynPPLUFf7x3@toroid.org>
References: 
 <CAOgcT0P0OGFfNhCKRUyV-rGMXk+JnsYUKGF3tPYhog2c1EQAmg@mail.gmail.com>
 <CA+TgmoY_kqYZdpuXLvcX9wj=8Lq6votwT-UJyMDE0btkOTyiWQ@mail.gmail.com>
 <CAOgcT0OeWuX+O=-5OTL8VWdjCaCh0HKDoKHDRq1zUfUrdXCnsw@mail.gmail.com>
 <CA+TgmoZvqk7UuzxsX1xjJRmMGkqoUGYTZLDCH8SmU1xTPr1Xig@mail.gmail.com>
 <CA+TgmobrOXbDh+hCzzVkD3weV3R-QRy3SPa=FRb_Rv9wF5iPJw@mail.gmail.com>
 <20211115162641.dmo6l32fklh64gnw@localhost>
 <CA+TgmoYScZ-wKtc+s_mLqSGGskd=vPb0Yh8ci_yQUgfNYG9rLQ@mail.gmail.com>
 <CA+TgmoaYZbz0=Yk797aOJwkGJC-LK3iXn+wzzMx7KdwNpZhS5g@mail.gmail.com>
 <CA+TgmoboJJE6M+u4f2E9vKaKGVq6jKync81Bi1jFW8QChscwOg@mail.gmail.com>
 <CA+TgmoYzYVwkDpwHru5+Lf5f0u4cTN6FeBLsJeHxXTBS7v=11A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CA+TgmoYzYVwkDpwHru5+Lf5f0u4cTN6FeBLsJeHxXTBS7v=11A@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/YgPEhynPPLUFf7x3%40toroid.org>
Precedence: bulk

At 2022-02-02 10:55:53 -0500, robertmhaas@gmail.com wrote:
>
> On Tue, Jan 18, 2022 at 1:55 PM Robert Haas <robertmhaas@gmail.com> wrote:
> > 0001 adds "server" and "blackhole" as backup targets. It now has some
> > tests. This might be more or less ready to ship, unless somebody else
> > sees a problem, or I find one.
> 
> I played around with this a bit and it seems quite easy to extend this
> further. So please find attached a couple more patches to generalize
> this mechanism.

It took me a while to assimilate these patches, including the backup
targets one, which I hadn't looked at before. Now that I've wrapped my
head around how to put the pieces together, I really like the idea. As
you say, writing non-trivial integrations in C will take some effort,
but it seems worthwhile. It's also nice that one can continue to use
pg_basebackup to trigger the backups and see progress information.

> Granted, coding up a new base backup target is
> something only experienced C hackers are likely to do, but the fact
> that I was able to throw this together so quickly suggests to me that
> I've got the design basically right, and that anyone who does want to
> plug into the new mechanism shouldn't have too much trouble doing so.
> 
> Thoughts?

Yes, it looks simple to follow the example set by basebackup_to_shell to
write a custom target. The complexity will be in whatever we need to do
to store/forward the backup data, rather than in obtaining the data in
the first place, which is exactly as it should be.

Thanks!

-- Abhijit