Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vjB16-009XnO-2v for pgsql-hackers@arkaria.postgresql.org; Fri, 23 Jan 2026 06:51:37 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1vjB15-00GRdc-1i for pgsql-hackers@arkaria.postgresql.org; Fri, 23 Jan 2026 06:51:35 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vjB15-00GRdS-0F for pgsql-hackers@lists.postgresql.org; Fri, 23 Jan 2026 06:51:35 +0000 Received: from mail-ed1-x535.google.com ([2a00:1450:4864:20::535]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1vjB11-001tJz-0H for pgsql-hackers@lists.postgresql.org; Fri, 23 Jan 2026 06:51:33 +0000 Received: by mail-ed1-x535.google.com with SMTP id 4fb4d7f45d1cf-64b92abe63aso4140068a12.0 for ; Thu, 22 Jan 2026 22:51:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1769151089; cv=none; d=google.com; s=arc-20240605; b=HBeTrn8hZcMA+1Iz7PWISANK68UUztftjMT0cOw3i251Y4pTruI4qU0rKiGp0YRf88 01UJsIFz+MxnFt3rNya24AtdIlMt2Li9LKixmi9eTStRsGlk64/IYxMsZpd/FoUPAN3p KQ9MmlUse230wNKfXeGkYKTeujX6jRHOVU6TjC/MBwunZcyyu3/8JR5vlHQIPlqdlC2i KABYWP24R0EQFrL+2SBY8YQNEVQz1ShvRfFYeCq2venmwPJWVegb8f66ENFmPPQUUVyg b8XXLqO6JG1QAiIWXUX5gKZ6HhhU57eBkeFJObwhBc2mkgyvgu7nvv6YWyMzLP7ZVOFQ tm2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=Kpg9FqlqW8Upear+ZDghE0x+IbGINJ64vEINYCnTDNw=; fh=psQVFdJFd37H5WCm6/cqLu9D4bbjW0fIMQSQyseIpMs=; b=UrYHoW+Ro6U7SQO5clwHTF4a4f4wE6jdWKLKZrExU7cPq0fnK1nVN7DYdRd3Ecjl5H jS1Wg9KFxUlUEXwl73j2ReqqmAEomKaMZg8VXvoWlhoy7OKpG+QZuKpJnEN2V+7oIlZt E6ggMqlwrB28OCNKq7sfhCa4zSoMCfuuxYSqIGCXLXITlBrM8UYoYhBaWVyY+KEJ6OUT bSsH5f6+X9+qLypt6S+FkNiTMsqSO61d/o/rKFkUbPHJBLcGBgoXpNRyv3KIM3FHqu7+ X9JxQAt0YdgWkaWROhHVoyUF2fC9OjjjizCcgSMSlENxTAZDJMZoph+NqlYkevpTwOLf c2YA==; darn=lists.postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb.com; s=google; t=1769151089; x=1769755889; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Kpg9FqlqW8Upear+ZDghE0x+IbGINJ64vEINYCnTDNw=; b=Ho/K/45HGG1sl5zO2XKlVk/NtdPAKNLCFvjmneSKSVLfaxyLVsH/GfaZwwkAZ+OhJR q8xE3aJ2Sp9mt4GNoGwmStyWnLiS3PcKA9FQNdO8IxMFpbNauBC6TBfuq/4MhNec6Nra w9XVGQtgAet6Z0/O/tS70d7PFSErkUmZ2zdyz2Dj2nBW5SH4y6+NrB9QHit5CVafgETA fi6xf1MUrif9x2RXmfKiFTuF5gZcPrK4VKf5W9klgfPUEPRoBv1rv+XqEKZPACWV6Top LInDcyKvcj+7+9WMOYFxr9TChs5ULeisRGS1lGJaqsZF97TKyXdhwutUD9XGcxsO8q0/ uXTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769151089; x=1769755889; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Kpg9FqlqW8Upear+ZDghE0x+IbGINJ64vEINYCnTDNw=; b=bMn8Cnm2dezjm65KKIUL28iguIjb4m++RnGYPM/kfd4vux2Zf7tfx4srgeKibYVfps g0iLzJOQScC9Of+ooB0XWvKPyiAm1EnsmdeME4cczYJZZ5AB3IL6DoTkuhTHbDL1L+DU egEJKNBSFa/W4byQIia0OihFVuOXn6EE2pCJUSFee0UGIfydigKqDC4+i3YIFuX57QuB q2NF33ponmhxRH/fLz1dBGVXmH/hG8AkbM2kJILCJTELULzeBK6kXPJk8e4F8rahQKhe nw8F5R9Y8y4+4vbsG50LhYoXOkj2oP80uLAPTGZ7im4UJKPPIhGQ7N6DiPttzzKpEH+j YHBA== X-Forwarded-Encrypted: i=1; AJvYcCVIyp7sqaybSWJpRXT3e0DZIINkXRBDUkoe6QbZE8N8eypmLl1NiDS229p54NMoX2mj2/n4dMP6lnyXDmi8@lists.postgresql.org X-Gm-Message-State: AOJu0YyW13xDn/oFFq2+lk82ftyECb17SVqktMbvCQ1YOucxyCbX2/Dt 9wJjx1WSDL0UIDqIGALDiUrXUlny0OpjbXSHmewABH1PsRrqRTu/S3gXhbKz/v9ex561slyFYOW Keg/b4QemKIr2+g7G0+kAyGuIDum2UNjvuh0JlzY+ X-Gm-Gg: AZuq6aKF64itORYiqGHAxNBwpgjVn+EOixGmugm6wCDuYB5P82cKwF3tAdKM0Xy+BB6 KCtRqBNSWzXmLqyQ2VAC8+uClRX7Ip+RXdBT4/WlPRV56Qx0T5ccpT11W59L+JKaJdnSlnTHVPa RIwLWxygW3JTkSCcumq3D5yY/hHFhnSDnxdKDwtNXYnCdqQ1iQsFf7J4r4A5B0yg+uJENiYnfz1 ckdFmHeGeR9LUH416MuR6kkM+HhT0Szip2PQJYyugHGUT+iq6a7owQmN1H5q+XG6Ok9c+wqhA== X-Received: by 2002:a17:907:6d24:b0:b80:3738:2f11 with SMTP id a640c23a62f3a-b88670e40f9mr23744866b.13.1769151088954; Thu, 22 Jan 2026 22:51:28 -0800 (PST) MIME-Version: 1.0 References: <3f22a8bb-29e8-40cc-97a1-309181da2c13@dunslane.net> <2bed001a-462c-42da-9a6b-3c7884502932@dunslane.net> <20250824010811.4d.nmisch@google.com> <82eb35b8-7f07-493b-b689-0934919e1dc3@dunslane.net> <17555e46-4fb2-4265-90e0-95cb8ed584a6@dunslane.net> In-Reply-To: From: tushar Date: Fri, 23 Jan 2026 12:21:17 +0530 X-Gm-Features: AZwV_QiprRdjNNcSNAGW27SBobpU0Hge6Xk6qb4dcP8fiD9xq-v0GUEul2ynZeQ Message-ID: Subject: Re: Non-text mode for pg_dumpall To: Mahendra Singh Thalor Cc: jian he , Vaibhav Dalvi , pgsql-hackers@lists.postgresql.org Content-Type: multipart/alternative; boundary="00000000000036238c06490896b9" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --00000000000036238c06490896b9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Jan 17, 2026 at 1:39=E2=80=AFAM Mahendra Singh Thalor wrote: > Thanks Tushar for the testing. > > On Wed, 7 Jan 2026 at 13:53, tushar wrote= : > > > > > > > > On Tue, Jan 6, 2026 at 11:56=E2=80=AFAM Mahendra Singh Thalor < > mahi6run@gmail.com> wrote: > >> > >> > >> > >> > >> We have another thread for this. We have patches also. Last year, we > >> planned to block these databases at creation time. > >> > >> > > >> > It's probably harmless, we connect to the databases further down to > do actual work. But it's also not nice. The toc.glo seems to have a bunch > of extraneous entries of type COMMENT and CONNECT. Why is that? As far as > poible this should have output pretty much identical to a plain pg_dumpa= ll. > >> > > >> > > >> > cheer > >> > > >> > > >> > andrew > >> > > Thanks Andrew for the feedback. > > In the attached patch, I fixed some comments. In the next version, I > will try to make it much identical to a plain pg_dumpall. > > >> If we don't dump those comments in non-text format, then the output of > >> "pg_restore -f filename dump_non_text" will not be the same as the > >> plain dump of pg_dumpall. > >> > >> Here, I am attaching an updated patch for the review and testing. > >> > > > > Hi Mahendra, > > > > I found a scenario in which the table is not restored if > --transaction-size switch is used at the time of pg_restore operation > > > > Please refer this scenario: > > Case A --pg_restore operation with "--transaction-size" against the > dump (taken using pg_dump) - > > create a table ( create table t(n int); ) > > perform pg_dump ( ./pg_dump -Ft postgres -f xyz.tar) > > create a database (create database test;) > > perform pg_restore using switch "--transaction-size" ( ./pg_restore > --transaction-size=3D1 -d test xyz.tar) > > table is restored into test database > > > > Case B --pg_restore operation with "--transaction-size" against the > dump (taken using pg_dumpall) - > > create a table ( create table t(n int); ) > > perform pg_dumpall ( ./pg_dumpall -Ft -f abc.tar) > > create a new cluster, start the server against a different port > > perform pg_restore using switch "--transaction-size" (./pg_restore -Ft > --transaction-size=3D10 -d postgres abc.tar -p 9000 -C) > > table is not restored > > > > if i remove --transaction-size switch then this works. > > > > regards, > > > > Fixed. > > On Mon, 12 Jan 2026 at 13:39, tushar > wrote: > > > > > > > > On Tue, Jan 6, 2026 at 11:56=E2=80=AFAM Mahendra Singh Thalor < > mahi6run@gmail.com> wrote: > >> > >> > >> Here, I am attaching an updated patch for the review and testing. > >> > >> Note: some of the review comments are still not fixed. I am working on > >> those and will post an updated patch. > >> > > Hi Mahendra, > > Please refer this scenario - if we are using with "--jobs" switch then > getting an error at the time of restore > > > > Create a table ( create table t(n int); insert into t values (1); ) > > Perform pg_dumpall ( ./pg_dumpall -Fd -f abc1.dr ) > > Create a new cluster, start the server against a different port > > Perform pg_restore using switch "--jobs 4 " (./pg_restore -j 4 -d > postgres abc1.dr/ -p 9000 -C ) > > > > " > > [edb@1a1c15437e7c bin]$ ./pg_restore -j 4 -d postgres abc1.dr/ -p 9000 > -C > > pg_restore: error: could not execute query: ERROR: role "edb" already > exists > > Command was: CREATE ROLE edb; > > ALTER ROLE edb WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN > REPLICATION BYPASSRLS; > > > > > > pg_restore: error: could not execute query: ERROR: syntax error at or > near "\" > > LINE 1: \connect template1 > > ^ > > Command was: \connect template1 > > > > > > > > pg_restore: error: could not execute query: ERROR: syntax error at or > near "\" > > LINE 1: \connect postgres > > ^ > > Command was: \connect postgres > > > > > > > > pg_restore: warning: errors ignored on restore: 3 > > [edb@1a1c15437e7c bin]$ > > " > > > > regards, > > Fixed this syntax error but user error is still there for parallel > mode(for non-parallel, fixed). This will be fixed in the next version. > > Here, I am attaching an updated patch for the review and testing. > > Thanks Mahendra, a minor observation - The pg_restore output shows a double slash in the map.dat path (e.g., abc.tar//map.dat). While it doesn't break the restore, we may want to clean up the path joining logic. [edb@1a1c15437e7c bin]$ ./pg_restore -Ft -C abc.tar/ -d postgres -p 9011 -U ed -v pg_restore: found database "template1 " (OID: 1) in file "abc.tar//map.dat" pg_restore: found database "postgres " (OID: 5) in file "abc.tar//map.dat" regards, --00000000000036238c06490896b9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, Jan 17,= 2026 at 1:39=E2=80=AFAM Mahendra Singh Thalor <mahi6run@gmail.com> wrote:
Thanks Tushar for the testing.

On Wed, 7 Jan 2026 at 13:53, tushar <tushar.ahuja@enterprisedb.com> wrote= :
>
>
>
> On Tue, Jan 6, 2026 at 11:56=E2=80=AFAM Mahendra Singh Thalor <mahi6run@gmail.com&= gt; wrote:
>>
>>
>>
>>
>> We have another thread for this. We have patches also. Last year, = we
>> planned to block these databases at creation time.
>>
>> >
>> > It's probably harmless, we connect to the databases furth= er down to do actual work. But it's also not nice. The toc.glo seems to= have a bunch of extraneous entries of type COMMENT and CONNECT. Why is tha= t? As far as poible this should have output pretty much=C2=A0 identical to = a plain pg_dumpall.
>> >
>> >
>> > cheer
>> >
>> >
>> > andrew
>>

Thanks Andrew for the feedback.

In the attached patch, I fixed some comments. In the next version, I
will try to make it much identical to a plain pg_dumpall.

>> If we don't dump those comments in non-text format, then the o= utput of
>> "pg_restore -f filename dump_non_text" will not be the s= ame as the
>> plain dump of pg_dumpall.
>>
>> Here, I am attaching an updated patch for the review and testing.<= br> >>
>
> Hi Mahendra,
>
> I found a scenario=C2=A0 in which the table is not restored if --trans= action-size switch is used=C2=A0 at the time of pg_restore operation
>
> Please refer this scenario:
> Case A --pg_restore operation with "--transaction-size" agai= nst the=C2=A0 dump (taken using pg_dump) -
> create a table ( create table t(n int); )
> perform pg_dump ( ./pg_dump -Ft postgres -f xyz.tar)
> create a database (create database test;)
> perform pg_restore using switch "--transaction-size" ( ./pg_= restore --transaction-size=3D1 -d test xyz.tar)
> table is restored into test database
>
> Case B --pg_restore operation with "--transaction-size" agai= nst the=C2=A0 dump (taken using pg_dumpall) -
> create a table ( create table t(n int); )
> perform pg_dumpall ( ./pg_dumpall -Ft -f abc.tar)
> create a new cluster, start the server against a different port
> perform pg_restore using switch "--transaction-size" (./pg_r= estore -Ft --transaction-size=3D10 -d postgres abc.tar -p 9000 -C)
> table is not restored
>
> if i remove --transaction-size switch then this works.
>
> regards,
>

Fixed.

On Mon, 12 Jan 2026 at 13:39, tushar <tushar.ahuja@enterprisedb.com> wrot= e:
>
>
>
> On Tue, Jan 6, 2026 at 11:56=E2=80=AFAM Mahendra Singh Thalor <mahi6run@gmail.com&= gt; wrote:
>>
>>
>> Here, I am attaching an updated patch for the review and testing.<= br> >>
>> Note: some of the review comments are still not fixed. I am workin= g on
>> those and will post an updated patch.
>>
> Hi Mahendra,
> Please refer this scenario - if we are using with "--jobs" s= witch then getting an error at the time of restore
>
> Create a table (=C2=A0 create table t(n int); insert into t values (1)= ;=C2=A0 )
> Perform pg_dumpall ( ./pg_dumpall -Fd -f abc1.dr )
> Create a new cluster, start the server against a different port
> Perform pg_restore using switch "--jobs 4 " (./pg_restore -j= 4 -d postgres abc1.dr/=C2=A0 -p 9000 -C )
>
> "
> [edb@1a1c15437e7c bin]$ ./pg_restore -j 4 -d postgres abc1.dr/=C2=A0 -= p 9000 -C
> pg_restore: error: could not execute query: ERROR:=C2=A0 role "ed= b" already exists
> Command was: CREATE ROLE edb;
> ALTER ROLE edb WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN REPLIC= ATION BYPASSRLS;
>
>
> pg_restore: error: could not execute query: ERROR:=C2=A0 syntax error = at or near "\"
> LINE 1: \connect template1
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0^
> Command was: \connect template1
>
>
>
> pg_restore: error: could not execute query: ERROR:=C2=A0 syntax error = at or near "\"
> LINE 1: \connect postgres
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0^
> Command was: \connect postgres
>
>
>
> pg_restore: warning: errors ignored on restore: 3
> [edb@1a1c15437e7c bin]$
> "
>
> regards,

Fixed this syntax error but user error is still there for parallel
mode(for non-parallel, fixed). This will be fixed in the next version.

Here, I am attaching an updated patch for the review and testing.


Thanks Mahendra, a minor=C2=A0 observation -= =C2=A0=C2=A0The pg_restore output shows a double slash in the map.dat path = (e.g., abc.tar//map.dat).=C2=A0
While it doesn't break the re= store, we may want to clean up the path joining logic.

=
[edb@1a1c15437e7c bin]$ ./pg_restore -Ft -C abc.tar/ -d postgres -p 90= 11 =C2=A0-U =C2=A0ed -v
pg_restore: found database "template1
&= quot; (OID: 1) in file "abc.tar//map.dat"
pg_restore: found da= tabase "postgres
" (OID: 5) in file "abc.tar//map.dat&quo= t;

regards,
--00000000000036238c06490896b9--