Date: Tue, 14 Jan 2020 23:14:49 +0100
From: David Fetter <david@fetter.org>
To: Stephen Frost <sfrost@snowman.net>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Robert Haas <robertmhaas@gmail.com>,
	David Steele <david@pgmasters.net>,
	Tels <nospam-pg-abuse@bloodgate.com>,
	Suraj Kharage <suraj.kharage@enterprisedb.com>,
	Rushabh Lathia <rushabh.lathia@gmail.com>,
	Andrew Dunstan <andrew.dunstan@2ndquadrant.com>,
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>,
	Jeevan Chalke <jeevan.chalke@enterprisedb.com>,
	vignesh C <vignesh21@gmail.com>
Subject: Re: backup manifests
Message-ID: <20200114221449.GH32763@fetter.org>
References: <20200103170123.GX3195@tamriel.snowman.net>
 <CA+TgmobjkosrwPaEA6nxhC-JoZ4JGXSiQ+27Zkm3iMY0VnMwvQ@mail.gmail.com>
 <20200103193559.GY3195@tamriel.snowman.net>
 <CA+TgmoY1EpREir7QvAHBzyj370-+bmfFV2F_X3ZDytpxf=iezw@mail.gmail.com>
 <20200108013348.GF3195@tamriel.snowman.net>
 <00b28009-3781-4c4a-174b-be2c1bc52f27@pgmasters.net>
 <CA+TgmoYdEEtO9HEPdDjV8DUT+=haZf_3bsMEuVgkOWTjt+PSKA@mail.gmail.com>
 <12445.1579024384@sss.pgh.pa.us>
 <20200114183312.GF32763@fetter.org>
 <20200114203540.GC3195@tamriel.snowman.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200114203540.GC3195@tamriel.snowman.net>
User-Agent: Mutt/1.9.4 (2018-02-28)
Precedence: bulk

On Tue, Jan 14, 2020 at 03:35:40PM -0500, Stephen Frost wrote:
> Greetings,
> 
> * David Fetter (david@fetter.org) wrote:
> > On Tue, Jan 14, 2020 at 12:53:04PM -0500, Tom Lane wrote:
> > > Robert Haas <robertmhaas@gmail.com> writes:
> > > > ... I would also expect that depending on an external package
> > > > would provoke significant opposition. If we suck the code into core,
> > > > then we have to keep it up to date with the upstream, which is a
> > > > significant maintenance burden - look at all the time Tom has spent on
> > > > snowball, regex, and time zone code over the years.
> > > 
> > > Also worth noting is that we have a seriously bad track record about
> > > choosing external packages to depend on.  The regex code has no upstream
> > > maintainer anymore (well, the Tcl guys seem to think that *we* are
> > > upstream for that now), and snowball is next door to moribund.
> > > With C not being a particularly hip language to develop in anymore,
> > > it wouldn't surprise me in the least for any C-code JSON parser
> > > we might pick to go dead pretty soon.
> > 
> > Given jq's extreme popularity and compatible license, I'd nominate that.
> 
> I don't think that really changes Tom's concerns here about having an
> "upstream" for this.
> 
> For my part, I don't really agree with the whole "we don't want two
> different JSON parsers" when we've got two of a bunch of stuff between
> the frontend and the backend, particularly since I don't really think
> it'll end up being *that* much code.
> 
> My thought, which I had expressed to David (though he obviously didn't
> entirely agree with me since he suggested the other options), was to
> adapt the pgBackRest JSON parser, which isn't really all that much code.
> 
> Frustratingly, that code has got some internal pgBackRest dependency on
> things like the memory context system (which looks, unsurprisingly, an
> awful lot like what is in PG backend), the error handling and logging
> systems (which are different from PG because they're quite intentionally
> segregated from each other- something PG would benefit from, imv..), and
> Variadics (known in the PG backend as Datums, and quite similar to
> them..).

It might be more fun to put in that infrastructure and have it gate
the manifest feature than to have two vastly different parsers to
contend with. I get that putting off the backup manifests isn't an
awesome prospect, but neither is rushing them in and getting them
wrong in ways we'll still be regretting a decade hence.

Best,
David.
-- 
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate