Content-Type: multipart/alternative;
 boundary="------------n4Nodexf6F7aMIz9Q6BsqFjU"
Message-ID: <82eb35b8-7f07-493b-b689-0934919e1dc3@dunslane.net>
Date: Sun, 24 Aug 2025 12:42:56 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Non-text mode for pg_dumpall
To: Noah Misch <noah@leadboat.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Mahendra Singh Thalor <mahi6run@gmail.com>,
 =?UTF-8?Q?=C3=81lvaro_Herrera?= <alvherre@alvh.no-ip.org>,
 jian he <jian.universality@gmail.com>, Srinath Reddy
 <srinath2133@gmail.com>, pgsql-hackers@lists.postgresql.org
References: <3f22a8bb-29e8-40cc-97a1-309181da2c13@dunslane.net>
 <20250722005339.ca.nmisch@google.com>
 <a5359ad4-4e2b-432d-92c0-188f9d558f06@dunslane.net>
 <20250725162141.6f.nmisch@google.com>
 <d1bfd0e7-f70c-48ef-ad5c-ac9d251e4b4c@dunslane.net>
 <2225040.1753477169@sss.pgh.pa.us> <20250727235628.e2.nmisch@google.com>
 <8f2ad50b-ebb7-4adc-997e-25e0ad96ff34@dunslane.net>
 <2bed001a-462c-42da-9a6b-3c7884502932@dunslane.net>
 <fd25e8b7-9538-4e44-90e5-8a357ac03eff@dunslane.net>
 <20250824010811.4d.nmisch@google.com>
From: Andrew Dunstan <andrew@dunslane.net>
Content-Language: en-US
Autocrypt: addr=andrew@dunslane.net; keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNJEFuZHJldyBEdW5zdGFuIDxhbmRyZXdAZHVuc2xhbmUubmV0PsLAlwQTAQgAQQIbAwIX
 gAIZAQULCQgHAwUVCgkICwUWAgMBAAIeBRYhBOQ+WEYd/Hy/RGkVpZn6f8tZ/DuBBQJoGNGd
 BQkdEO8nAAoJEJn6f8tZ/DuBq74H/jkTR4Zi3stbw+xC7v2u3QozssK7MYPL2AsVfh7OealS
 h182fiWXpfvmmAB7WUHbhk9GC2RAOnHI/2d2jgKaMLAHsGYOT0YopTVIwRY43fCw/mK67yxc
 wmDcX+zyKfLaivNbf5A7QPLNwda98bEAMSJ8Sn652Uc6cA8t3uKGsVzbRBQOoYzjgvBCfSrE
 9ql3PDNg0l4BfAqabd2f70ZUm9VAMEPrgv/v2xI7M2XiL4g5BVmqLCOwxLM8RMCotCuoweUr
 VO43DeBCIDwLxotMJKvGWDjBzQYlU1NPUAtNcz/gN9ITUe1VUGjyvGj4u1lxBOcQQUw7l1+T
 5moZ4iZxXzvOwE0ETspYWQEIANGc4zQULOxhbqO2dyD51YhqCNRmm9oKWaqf+wmW4tpDe/VV
 cxAnNizd4LWCHfzpb5cHAtGkOPePMfzWVf6nvdF7d3eglbtf59+zG7O7llV0xSSoFiieQBsr
 GvqDInXYX/4mRRXMtyhM353/tixC9RWLs1oofyYmCPPXXY7h9R7en3B8BoVrRFcdzlIY/NFN
 hFGW/9dkEiGjgna2Rk6e15kln4ZvFBWUg23p93w/pqXcxY6+k/8TEk+C4R+M6w7o2PLGOjdZ
 +kPiUcw5H85zf/yZJwQXzisXaNduwWB6Vads9YC9dj6kPR1c4VGRqAaYL++LAEOqrlvm2Tvq
 QqZRtnEAEQEAAcLAfAQYAQgAJgIbDBYhBOQ+WEYd/Hy/RGkVpZn6f8tZ/DuBBQJoGNI2BQkd
 EODdAAoJEJn6f8tZ/DuBfw0IAKTsfD40teP/pp+bsLLMSxPXUYrrprTj7WFB5v61p6dkpSr/
 qXmMlyahdxQFaPmfVgVirB1Vk/kHiWNnnGjfUV9nB2Zg9LI0Xb9/ts3LsUiRWXzG3tkMY6XL
 vsVOxW4XFRND9l2q+WW93aZ1DZl+fqWfYgMvsusFRhmGFOKTRfKPta2Pkv+AhA24N4+PrR5p
 bU4k2MO8PAGiK8eaYKGFG1bHKuAvoDoF7WXJ3FHxuWqLnKEt4dfOLm5pAe3zq1Lt6q8azT9i
 QWGpSAK5vQUWQHBHpiDjdPeqKZ6HiAXIIKfSmb+jrvXBqoP+D6/K7rUjG2aXiRtTIAXms9sm
 VRu7cmw=
In-Reply-To: <20250824010811.4d.nmisch@google.com>
Archived-At: 
 <https://www.postgresql.org/message-id/82eb35b8-7f07-493b-b689-0934919e1dc3%40dunslane.net>
Precedence: bulk

This is a multi-part message in MIME format.
--------------n4Nodexf6F7aMIz9Q6BsqFjU
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit


On 2025-08-23 Sa 9:08 PM, Noah Misch wrote:
> On Wed, Jul 30, 2025 at 02:51:59PM -0400, Andrew Dunstan wrote:
>> OK, now that's reverted we should discuss how to proceed. I had two thoughts
>> - we could use invent a JSON format for the globals, or we could just use
>> the existing archive format. I think the archive format is pretty flexible,
>> and should be able to accommodate this. The downside is it's not humanly
>> readable. The upside is that we don't need to do anything special either to
>> write it or parse it.
> I would first try to use the existing archiver API, because that makes it
> harder to miss bugs.  Any tension between that API and pg_dumpall is likely to
> have corresponding tension on the pg_restore side.  Resolving that tension
> will reveal much of the project's scope that remained hidden during the v18
> attempt.  Perhaps more important than that, using the archiver API means
> future pg_dump and pg_restore options are more likely to cooperate properly
> with $SUBJECT.  In other words, I want it to be hard to add pg_dump/pg_restore
> features that malfunction only for $SUBJECT archives.  The strength of the
> archiver architecture shows in how rarely new features need format-specific
> logic and how rarely format-specific bugs get reported.  We've had little or
> no trouble with e.g. bugs that appear in -Fd but not in -Fc.


Yeah, that's what we're going to try.


cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

--------------n4Nodexf6F7aMIz9Q6BsqFjU
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 2025-08-23 Sa 9:08 PM, Noah Misch
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:20250824010811.4d.nmisch@google.com">
      <pre wrap="" class="moz-quote-pre">On Wed, Jul 30, 2025 at 02:51:59PM -0400, Andrew Dunstan wrote:
</pre>
      <blockquote type="cite">
        <pre wrap="" class="moz-quote-pre">OK, now that's reverted we should discuss how to proceed. I had two thoughts
- we could use invent a JSON format for the globals, or we could just use
the existing archive format. I think the archive format is pretty flexible,
and should be able to accommodate this. The downside is it's not humanly
readable. The upside is that we don't need to do anything special either to
write it or parse it.
</pre>
      </blockquote>
      <pre wrap="" class="moz-quote-pre">
I would first try to use the existing archiver API, because that makes it
harder to miss bugs.  Any tension between that API and pg_dumpall is likely to
have corresponding tension on the pg_restore side.  Resolving that tension
will reveal much of the project's scope that remained hidden during the v18
attempt.  Perhaps more important than that, using the archiver API means
future pg_dump and pg_restore options are more likely to cooperate properly
with $SUBJECT.  In other words, I want it to be hard to add pg_dump/pg_restore
features that malfunction only for $SUBJECT archives.  The strength of the
archiver architecture shows in how rarely new features need format-specific
logic and how rarely format-specific bugs get reported.  We've had little or
no trouble with e.g. bugs that appear in -Fd but not in -Fc.</pre>
    </blockquote>
    <p><br>
    </p>
    <p>Yeah, that's what we're going to try.</p>
    <p><br>
    </p>
    <p>cheers</p>
    <p><br>
    </p>
    <p>andrew<br>
    </p>
    <p><span style="white-space: pre-wrap">
</span></p>
    <pre class="moz-signature"
    signature-switch-id="d0437855-2267-4610-80b3-83167ec45b0b" cols="72">--
Andrew Dunstan
EDB: <a class="moz-txt-link-freetext" href="https://www.enterprisedb.com">https://www.enterprisedb.com</a>
</pre>
  </body>
</html>

--------------n4Nodexf6F7aMIz9Q6BsqFjU--