public inbox for [email protected]
help / color / mirror / Atom feedFrom: tushar <[email protected]>
To: Mahendra Singh Thalor <[email protected]>
Cc: jian he <[email protected]>
Cc: Vaibhav Dalvi <[email protected]>
Cc: [email protected]
Subject: Re: Non-text mode for pg_dumpall
Date: Fri, 23 Jan 2026 12:21:17 +0530
Message-ID: <CAC6VRoZbP=-=a+a78RoWcs2L4=4VEDZqJMg+7SyUAcSHATyDAQ@mail.gmail.com> (raw)
In-Reply-To: <CAKYtNApq+6Riz26So1nmXkRpL_7o868+r6i=wxQ+AgGQ4oSLig@mail.gmail.com>
References: <[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<CAKYtNAr+OMjcGKwd+AgWA+s=8EWGtg5PkVV3O7X6d3eCv=MUeQ@mail.gmail.com>
<CAKYtNApzLLeCqt5fHDzZOTnzCdCnBt3Y_fytFmJ0LMNHDPY-yA@mail.gmail.com>
<CAKYtNArSYJdjez541C0qPZ9R3_yzPDpScAd=-RU4kPRCJh4viQ@mail.gmail.com>
<CAKYtNArGUTzrfTBxpftL_yAgkKE+hcDgXVfQvjB2HFO9rGhE5g@mail.gmail.com>
<CA+vB=AEQc3Xqz+KMh35Zsa-SsRP-n=HXGT4RwwNou4-__Prx+A@mail.gmail.com>
<CAKYtNAqJqDmKcqCzpHg2SO=2MTxvE7rOWCACsoWsO7520tUWKw@mail.gmail.com>
<CA+vB=AE9ypeQV-hhXscJ6T75BrAFfZh56GX_eM=eVG2n1o_Beg@mail.gmail.com>
<CA+vB=AGsn4eUxsbLk_oy=iKzd8D_1Ne375XH-2u6Zncu72Q01Q@mail.gmail.com>
<CAKYtNAqQ5LHWNWyaeVOJaS=2xDJovfv9GShTBzf8_5s=jH7wsg@mail.gmail.com>
<CAKYtNAoSTBqR24cn3XDOnwk2DCL+nAUkjpB5Xkz1H74rDqa-aQ@mail.gmail.com>
<[email protected]>
<CAKYtNAoz9JZC+-Uo4QsA1O0PkEehxhTFy1y3HJjqzv6jwrXvCQ@mail.gmail.com>
<CA+vB=AETksQZpjyBosrZv6N5A6DjaCtMQop3+MB8GDj0XnYoxQ@mail.gmail.com>
<CAKYtNAoEUvYEG207zaGY0pEh6TB2sk6hpuz9LdG-fYEC=e2CgQ@mail.gmail.com>
<CAKYtNAppx6y7M+9Fb5+TtTNhRS6onZo9kHeQS6yQF7PiK6qnEA@mail.gmail.com>
<CAC6VRoZie9-iC9gWcJs3QGx=2_tA34fMBmj7Ck0VAP2JpDJGug@mail.gmail.com>
<CAC6VRoZHZWzHZ0Oh++WW=9Q83co4AyG6u4+7HVi1gJiuv1CZ2g@mail.gmail.com>
<CAC6VRoYWPQZeLDuKVNeMz5z-iiVnv0zWyBwWhxdm4UQO17-AUQ@mail.gmail.com>
<CAKYtNAoNi8O1wbSZx0a1oSi-YzQtprxqpfZqryRq39hXn_p-OA@mail.gmail.com>
<CAC6VRoYCqJq=0q3GF+vfDVbY4JUehqsuxYhcmVi7A=1hfNzSJg@mail.gmail.com>
<CAKYtNApm8nA6hVMNkSjeNPiwTZBWYkhJKU7haQghUZdw74tYxA@mail.gmail.com>
<CACJufxHc5Rp2roGyJOVnCk=Mx8_42_9vCGJSCFpLB5=AFQqHTQ@mail.gmail.com>
<CAKYtNAoz-Sz2X1ipkQ5x1Sh08AMyy1kDRGfeRW3P1W-QoN+WOg@mail.gmail.com>
<CAC6VRoZU2L2GX7+bXpsZ6bo+NuxBOe1dTGDPzKobUL4KVdm46A@mail.gmail.com>
<CAKYtNApq+6Riz26So1nmXkRpL_7o868+r6i=wxQ+AgGQ4oSLig@mail.gmail.com>
On Sat, Jan 17, 2026 at 1:39 AM Mahendra Singh Thalor <[email protected]>
wrote:
> Thanks Tushar for the testing.
>
> On Wed, 7 Jan 2026 at 13:53, tushar <[email protected]> wrote:
> >
> >
> >
> > On Tue, Jan 6, 2026 at 11:56 AM Mahendra Singh Thalor <
> [email protected]> wrote:
> >>
> >>
> >>
> >>
> >> We have another thread for this. We have patches also. Last year, we
> >> planned to block these databases at creation time.
> >>
> >> >
> >> > It's probably harmless, we connect to the databases further down to
> do actual work. But it's also not nice. The toc.glo seems to have a bunch
> of extraneous entries of type COMMENT and CONNECT. Why is that? As far as
> poible this should have output pretty much identical to a plain pg_dumpall.
> >> >
> >> >
> >> > cheer
> >> >
> >> >
> >> > andrew
> >>
>
> Thanks Andrew for the feedback.
>
> In the attached patch, I fixed some comments. In the next version, I
> will try to make it much identical to a plain pg_dumpall.
>
> >> If we don't dump those comments in non-text format, then the output of
> >> "pg_restore -f filename dump_non_text" will not be the same as the
> >> plain dump of pg_dumpall.
> >>
> >> Here, I am attaching an updated patch for the review and testing.
> >>
> >
> > Hi Mahendra,
> >
> > I found a scenario in which the table is not restored if
> --transaction-size switch is used at the time of pg_restore operation
> >
> > Please refer this scenario:
> > Case A --pg_restore operation with "--transaction-size" against the
> dump (taken using pg_dump) -
> > create a table ( create table t(n int); )
> > perform pg_dump ( ./pg_dump -Ft postgres -f xyz.tar)
> > create a database (create database test;)
> > perform pg_restore using switch "--transaction-size" ( ./pg_restore
> --transaction-size=1 -d test xyz.tar)
> > table is restored into test database
> >
> > Case B --pg_restore operation with "--transaction-size" against the
> dump (taken using pg_dumpall) -
> > create a table ( create table t(n int); )
> > perform pg_dumpall ( ./pg_dumpall -Ft -f abc.tar)
> > create a new cluster, start the server against a different port
> > perform pg_restore using switch "--transaction-size" (./pg_restore -Ft
> --transaction-size=10 -d postgres abc.tar -p 9000 -C)
> > table is not restored
> >
> > if i remove --transaction-size switch then this works.
> >
> > regards,
> >
>
> Fixed.
>
> On Mon, 12 Jan 2026 at 13:39, tushar <[email protected]>
> wrote:
> >
> >
> >
> > On Tue, Jan 6, 2026 at 11:56 AM Mahendra Singh Thalor <
> [email protected]> wrote:
> >>
> >>
> >> Here, I am attaching an updated patch for the review and testing.
> >>
> >> Note: some of the review comments are still not fixed. I am working on
> >> those and will post an updated patch.
> >>
> > Hi Mahendra,
> > Please refer this scenario - if we are using with "--jobs" switch then
> getting an error at the time of restore
> >
> > Create a table ( create table t(n int); insert into t values (1); )
> > Perform pg_dumpall ( ./pg_dumpall -Fd -f abc1.dr )
> > Create a new cluster, start the server against a different port
> > Perform pg_restore using switch "--jobs 4 " (./pg_restore -j 4 -d
> postgres abc1.dr/ -p 9000 -C )
> >
> > "
> > [edb@1a1c15437e7c bin]$ ./pg_restore -j 4 -d postgres abc1.dr/ -p 9000
> -C
> > pg_restore: error: could not execute query: ERROR: role "edb" already
> exists
> > Command was: CREATE ROLE edb;
> > ALTER ROLE edb WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN
> REPLICATION BYPASSRLS;
> >
> >
> > pg_restore: error: could not execute query: ERROR: syntax error at or
> near "\"
> > LINE 1: \connect template1
> > ^
> > Command was: \connect template1
> >
> >
> >
> > pg_restore: error: could not execute query: ERROR: syntax error at or
> near "\"
> > LINE 1: \connect postgres
> > ^
> > Command was: \connect postgres
> >
> >
> >
> > pg_restore: warning: errors ignored on restore: 3
> > [edb@1a1c15437e7c bin]$
> > "
> >
> > regards,
>
> Fixed this syntax error but user error is still there for parallel
> mode(for non-parallel, fixed). This will be fixed in the next version.
>
> Here, I am attaching an updated patch for the review and testing.
>
>
Thanks Mahendra, a minor observation - The pg_restore output shows a
double slash in the map.dat path (e.g., abc.tar//map.dat).
While it doesn't break the restore, we may want to clean up the path
joining logic.
[edb@1a1c15437e7c bin]$ ./pg_restore -Ft -C abc.tar/ -d postgres -p 9011
-U ed -v
pg_restore: found database "template1
" (OID: 1) in file "abc.tar//map.dat"
pg_restore: found database "postgres
" (OID: 5) in file "abc.tar//map.dat"
regards,
view thread (6+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Non-text mode for pg_dumpall
In-Reply-To: <CAC6VRoZbP=-=a+a78RoWcs2L4=4VEDZqJMg+7SyUAcSHATyDAQ@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox