Date: Sat, 23 Aug 2025 18:08:11 -0700
From: Noah Misch <noah@leadboat.com>
To: Andrew Dunstan <andrew@dunslane.net>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
	Mahendra Singh Thalor <mahi6run@gmail.com>,
	=?iso-8859-1?Q?=C1lvaro?= Herrera <alvherre@alvh.no-ip.org>,
	jian he <jian.universality@gmail.com>,
	Srinath Reddy <srinath2133@gmail.com>,
	pgsql-hackers@lists.postgresql.org
Subject: Re: Non-text mode for pg_dumpall
Message-ID: <20250824010811.4d.nmisch@google.com>
References: <3f22a8bb-29e8-40cc-97a1-309181da2c13@dunslane.net>
 <20250722005339.ca.nmisch@google.com>
 <a5359ad4-4e2b-432d-92c0-188f9d558f06@dunslane.net>
 <20250725162141.6f.nmisch@google.com>
 <d1bfd0e7-f70c-48ef-ad5c-ac9d251e4b4c@dunslane.net>
 <2225040.1753477169@sss.pgh.pa.us>
 <20250727235628.e2.nmisch@google.com>
 <8f2ad50b-ebb7-4adc-997e-25e0ad96ff34@dunslane.net>
 <2bed001a-462c-42da-9a6b-3c7884502932@dunslane.net>
 <fd25e8b7-9538-4e44-90e5-8a357ac03eff@dunslane.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <fd25e8b7-9538-4e44-90e5-8a357ac03eff@dunslane.net>
User-Agent: Mutt/2.2.12 (2023-09-09)
Archived-At: 
 <https://www.postgresql.org/message-id/20250824010811.4d.nmisch%40google.com>
Precedence: bulk

On Wed, Jul 30, 2025 at 02:51:59PM -0400, Andrew Dunstan wrote:
> OK, now that's reverted we should discuss how to proceed. I had two thoughts
> - we could use invent a JSON format for the globals, or we could just use
> the existing archive format. I think the archive format is pretty flexible,
> and should be able to accommodate this. The downside is it's not humanly
> readable. The upside is that we don't need to do anything special either to
> write it or parse it.

I would first try to use the existing archiver API, because that makes it
harder to miss bugs.  Any tension between that API and pg_dumpall is likely to
have corresponding tension on the pg_restore side.  Resolving that tension
will reveal much of the project's scope that remained hidden during the v18
attempt.  Perhaps more important than that, using the archiver API means
future pg_dump and pg_restore options are more likely to cooperate properly
with $SUBJECT.  In other words, I want it to be hard to add pg_dump/pg_restore
features that malfunction only for $SUBJECT archives.  The strength of the
archiver architecture shows in how rarely new features need format-specific
logic and how rarely format-specific bugs get reported.  We've had little or
no trouble with e.g. bugs that appear in -Fd but not in -Fc.

If pg_backup_json.c emerged as a new backend to the archiver API, I'd not have
concerns about that.  But a JSON format specific to $SUBJECT sounds like a
recipe for bugs.

> There might also be other reasonable options. But I think we should stay out
> of the business of using custom code to parse text.

Agreed.