Content-Type: multipart/alternative;
 boundary="------------ZWF6U1aGYKhYsnxz2WYLUsgr"
Message-ID: <d1bfd0e7-f70c-48ef-ad5c-ac9d251e4b4c@dunslane.net>
Date: Fri, 25 Jul 2025 15:31:47 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Non-text mode for pg_dumpall
To: Noah Misch <noah@leadboat.com>
Cc: Mahendra Singh Thalor <mahi6run@gmail.com>,
 =?UTF-8?Q?=C3=81lvaro_Herrera?= <alvherre@alvh.no-ip.org>,
 jian he <jian.universality@gmail.com>, Srinath Reddy
 <srinath2133@gmail.com>, pgsql-hackers@lists.postgresql.org
References: 
 <CAKYtNAq530UTAtnVs5xfONQ0j6vS-Sys50p5+SNfC7G7_ghCVQ@mail.gmail.com>
 <948154fe-0278-4f4c-8f5a-085e12f03163@dunslane.net>
 <20250708212819.09.nmisch@google.com>
 <CAKYtNAr-6_CCz+tnJ0T2597Ar4iAZXkE1qPimsO9EY-Kn7LvzQ@mail.gmail.com>
 <20250716001957.c6.nmisch@google.com>
 <CAKYtNAqXTvfAw-y4FHvzprg72cFrC63cge9xMVDO_R4=NHc5rA@mail.gmail.com>
 <CAKYtNAppY=vVtTMdkxRPV48cHKyzLeYF6mrPk15Wczme7e9=ww@mail.gmail.com>
 <3f22a8bb-29e8-40cc-97a1-309181da2c13@dunslane.net>
 <20250722005339.ca.nmisch@google.com>
 <a5359ad4-4e2b-432d-92c0-188f9d558f06@dunslane.net>
 <20250725162141.6f.nmisch@google.com>
From: Andrew Dunstan <andrew@dunslane.net>
Content-Language: en-US
Autocrypt: addr=andrew@dunslane.net; keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNJEFuZHJldyBEdW5zdGFuIDxhbmRyZXdAZHVuc2xhbmUubmV0PsLAlwQTAQgAQQIbAwIX
 gAIZAQULCQgHAwUVCgkICwUWAgMBAAIeBRYhBOQ+WEYd/Hy/RGkVpZn6f8tZ/DuBBQJoGNGd
 BQkdEO8nAAoJEJn6f8tZ/DuBq74H/jkTR4Zi3stbw+xC7v2u3QozssK7MYPL2AsVfh7OealS
 h182fiWXpfvmmAB7WUHbhk9GC2RAOnHI/2d2jgKaMLAHsGYOT0YopTVIwRY43fCw/mK67yxc
 wmDcX+zyKfLaivNbf5A7QPLNwda98bEAMSJ8Sn652Uc6cA8t3uKGsVzbRBQOoYzjgvBCfSrE
 9ql3PDNg0l4BfAqabd2f70ZUm9VAMEPrgv/v2xI7M2XiL4g5BVmqLCOwxLM8RMCotCuoweUr
 VO43DeBCIDwLxotMJKvGWDjBzQYlU1NPUAtNcz/gN9ITUe1VUGjyvGj4u1lxBOcQQUw7l1+T
 5moZ4iZxXzvOwE0ETspYWQEIANGc4zQULOxhbqO2dyD51YhqCNRmm9oKWaqf+wmW4tpDe/VV
 cxAnNizd4LWCHfzpb5cHAtGkOPePMfzWVf6nvdF7d3eglbtf59+zG7O7llV0xSSoFiieQBsr
 GvqDInXYX/4mRRXMtyhM353/tixC9RWLs1oofyYmCPPXXY7h9R7en3B8BoVrRFcdzlIY/NFN
 hFGW/9dkEiGjgna2Rk6e15kln4ZvFBWUg23p93w/pqXcxY6+k/8TEk+C4R+M6w7o2PLGOjdZ
 +kPiUcw5H85zf/yZJwQXzisXaNduwWB6Vads9YC9dj6kPR1c4VGRqAaYL++LAEOqrlvm2Tvq
 QqZRtnEAEQEAAcLAfAQYAQgAJgIbDBYhBOQ+WEYd/Hy/RGkVpZn6f8tZ/DuBBQJoGNI2BQkd
 EODdAAoJEJn6f8tZ/DuBfw0IAKTsfD40teP/pp+bsLLMSxPXUYrrprTj7WFB5v61p6dkpSr/
 qXmMlyahdxQFaPmfVgVirB1Vk/kHiWNnnGjfUV9nB2Zg9LI0Xb9/ts3LsUiRWXzG3tkMY6XL
 vsVOxW4XFRND9l2q+WW93aZ1DZl+fqWfYgMvsusFRhmGFOKTRfKPta2Pkv+AhA24N4+PrR5p
 bU4k2MO8PAGiK8eaYKGFG1bHKuAvoDoF7WXJ3FHxuWqLnKEt4dfOLm5pAe3zq1Lt6q8azT9i
 QWGpSAK5vQUWQHBHpiDjdPeqKZ6HiAXIIKfSmb+jrvXBqoP+D6/K7rUjG2aXiRtTIAXms9sm
 VRu7cmw=
In-Reply-To: <20250725162141.6f.nmisch@google.com>
Archived-At: 
 <https://www.postgresql.org/message-id/d1bfd0e7-f70c-48ef-ad5c-ac9d251e4b4c%40dunslane.net>
Precedence: bulk

This is a multi-part message in MIME format.
--------------ZWF6U1aGYKhYsnxz2WYLUsgr
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit


On 2025-07-25 Fr 12:21 PM, Noah Misch wrote:
> On Thu, Jul 24, 2025 at 04:33:15PM -0400, Andrew Dunstan wrote:
>> On 2025-07-21 Mo 8:53 PM, Noah Misch wrote:
>>> I suspect this is going to end with a structured dump like we use on the
>>> pg_dump (per-database) side.  It's not an accident that v17 pg_restore doesn't
>>> lex text files to do its job.  pg_dumpall deals with a more-limited set of
>>> statements than pg_dump deals with, but they're not _that much_ more limited.
>>> I won't veto a lexing-based approach if it gets the behaviors right, but I
>>> don't have high hopes for it getting the behaviors right and staying that way.
>> I have been talking offline with Mahendra about this. I agree that we would
>> be better off with a structured object for globals. But the thing that's
>> been striking me all afternoon as I have pondered it is that we should not
>> be designing such an animal at this stage of the cycle. Whatever we do we're
>> going to be stuck supporting, so I have very reluctantly come to the
>> conclusion that it would probably be better to back the feature out and have
>> another go for PG 19.
> That makes sense to me.  It would be quite a sprint to get this done in time,
> and that wouldn't leave much room for additional testing and feedback before
> the final release.  I agree with the reluctance and with the conclusion.


Before we throw the baby out with the bathwater, how about this 
suggestion? pg_dumpall would continue to produce globals.dat, but it 
wouldn't be processed by pg_restore, which would only restore the 
individual databases. Or else we just don't produce globals.dat at all. 
Then we could introduce a structured object that pg_restore could safely 
use for release 19, and I think we'd still have something useful for 
release 18.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

--------------ZWF6U1aGYKhYsnxz2WYLUsgr
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 2025-07-25 Fr 12:21 PM, Noah Misch
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:20250725162141.6f.nmisch@google.com">
      <pre wrap="" class="moz-quote-pre">On Thu, Jul 24, 2025 at 04:33:15PM -0400, Andrew Dunstan wrote:
</pre>
      <blockquote type="cite">
        <pre wrap="" class="moz-quote-pre">On 2025-07-21 Mo 8:53 PM, Noah Misch wrote:
</pre>
        <blockquote type="cite">
          <pre wrap="" class="moz-quote-pre">I suspect this is going to end with a structured dump like we use on the
pg_dump (per-database) side.  It's not an accident that v17 pg_restore doesn't
lex text files to do its job.  pg_dumpall deals with a more-limited set of
statements than pg_dump deals with, but they're not _that much_ more limited.
I won't veto a lexing-based approach if it gets the behaviors right, but I
don't have high hopes for it getting the behaviors right and staying that way.
</pre>
        </blockquote>
        <pre wrap="" class="moz-quote-pre">
I have been talking offline with Mahendra about this. I agree that we would
be better off with a structured object for globals. But the thing that's
been striking me all afternoon as I have pondered it is that we should not
be designing such an animal at this stage of the cycle. Whatever we do we're
going to be stuck supporting, so I have very reluctantly come to the
conclusion that it would probably be better to back the feature out and have
another go for PG 19.
</pre>
      </blockquote>
      <pre wrap="" class="moz-quote-pre">
That makes sense to me.  It would be quite a sprint to get this done in time,
and that wouldn't leave much room for additional testing and feedback before
the final release.  I agree with the reluctance and with the conclusion.</pre>
    </blockquote>
    <p><br>
    </p>
    <p><br>
    </p>
    <p><span style="white-space: pre-wrap">Before we throw the baby out with the bathwater, how about this suggestion? pg_dumpall would continue to produce globals.dat, but it wouldn't be processed by pg_restore, which would only restore the individual databases. Or else we just don't produce globals.dat at all. Then we could introduce a structured object that pg_restore could safely use for release 19, and I think we'd still have something useful for release 18.</span></p>
    <p><span style="white-space: pre-wrap">
</span></p>
    <p><span style="white-space: pre-wrap">cheers</span></p>
    <p><span style="white-space: pre-wrap">
</span></p>
    <p><span style="white-space: pre-wrap">andrew
</span></p>
    <pre class="moz-signature"
    signature-switch-id="d0437855-2267-4610-80b3-83167ec45b0b" cols="72">--
Andrew Dunstan
EDB: <a class="moz-txt-link-freetext" href="https://www.enterprisedb.com">https://www.enterprisedb.com</a>
</pre>
  </body>
</html>

--------------ZWF6U1aGYKhYsnxz2WYLUsgr--