MIME-Version: 1.0
References: <2250061.1774104346@sss.pgh.pa.us>
 <2555285.1774131847@sss.pgh.pa.us>
 <2609460.1774153487@sss.pgh.pa.us>
 <CAD5tBcLVWKnph3iB-VPuPKR0dCckOJRFZW2-4H7HTTmhw8-vOg@mail.gmail.com>
 <2790913.1774200584@sss.pgh.pa.us> <2880042.1774203473@sss.pgh!!.pa.us>
 <CAD5tBcLsYDz+Nzx8MryjxiKaN3fGKd4ZgXuN1Jn=CYxw9dh+AA@mail.gmail.com>
 <3341199.1774221191@sss.pgh.pa.us> <3424809.1774234940@sss.pgh.pa.us>
 <acIA8KKBTv-T53YG@paquier.xyz>
 <x2tknjejjouleunkqrvpnwn2tuulunybinycidefm3wmnsyhht@pw5uo3wrqx43>
 <CA+hUKGL2dppjO4o28ZY7n_LTWviKLAi-7KZ=tx5w2HGevCEYPA@mail.gmail.com>
 <1624716.1774736283@sss.pgh.pa.us> <1626907.1774737417@sss.pgh.pa.us>
 <97a382c0-1f19-4ea0-951f-e37e6abc34a3@vondra.me>
 <1630755.1774739531@sss.pgh.pa.us>
 <c3a2b730-9d90-4a66-ae5e-50f6d77922d6@vondra.me>
 <CA+hUKGJyvdyWMC-RW1njqevD-q_gTbFq+DyDiFpUJVaG+DY20w@mail.gmail.com>
 <1873141.1774823011@sss.pgh.pa.us>
 <CA+hUKG+Pqz5=YQG_=8ho0YsTfn2HWOsJQWqS4j0q8QQWweJP9w@mail.gmail.com>
 <3049460.1775067940@sss.pgh.pa.us> <3118179.1775092964@sss.pgh.pa.us>
 <CA+hUKG+-pn14s_tjEBO6YKHmc=uRhGVn=w2oM91KKnEUc7pH0Q@mail.gmail.com>
 <3565835.1775147392@sss.pgh.pa.us>
 <CAA5RZ0tt89MgNi4-0F4onH+-TFSsysFjMM-tBc6aXbuQv5xBXw@mail.gmail.com>
In-Reply-To: 
 <CAA5RZ0tt89MgNi4-0F4onH+-TFSsysFjMM-tBc6aXbuQv5xBXw@mail.gmail.com>
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 3 Apr 2026 13:07:40 +1300
Message-ID: 
 <CA+hUKGLMkv_fnGXzVRO8qbx5uHs-qMn151GTJYCfn9w1ZamGNg@mail.gmail.com>
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
To: Sami Imseih <samimseih@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Tomas Vondra <tomas@vondra.me>,
	Andres Freund <andres@anarazel.de>, Michael Paquier <michael@paquier.xyz>,
	Andrew Dunstan <andrew@dunslane.net>, Amul Sul <sulamul@gmail.com>,
	Zsolt Parragi <zsolt.parragi@percona.com>,
 Robert Haas <robertmhaas@gmail.com>,
	Chao Li <li.evan.chao@gmail.com>,
	Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>,
 Fujii Masao <masao.fujii@gmail.com>,
	Jakub Wartak <jakub.wartak@enterprisedb.com>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: multipart/mixed; boundary="0000000000005066bb064e831d63"
Archived-At: 
 <https://www.postgresql.org/message-id/CA%2BhUKGLMkv_fnGXzVRO8qbx5uHs-qMn151GTJYCfn9w1ZamGNg%40mail.gmail.com>
Precedence: bulk

--0000000000005066bb064e831d63
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Apr 3, 2026 at 12:43=E2=80=AFPM Sami Imseih <samimseih@gmail.com> w=
rote:
> The --format=3Dustar has a limit of 2^21 (2097151) for UID/GID [1]
> and on my machine the UID is 10012663.
>
> So I found that one way to deal with this is to run the tar command with
> --owner=3D0 --group=3D0. As far as I can tell, the owner and group IDs do=
n't
> matter for these tests, so maybe that is OK.
>
> @@ -1333,6 +1333,10 @@ sub tar_portability_options
>                 =3D=3D 0)
>         {
>                 push(@tar_p_flags, "--format=3Dustar");
> +               # ustar format supports UIDs only up to 2^21 (2097151).
> +               # Override owner/group to avoid failures on systems where
> +               # the running user's UID/GID exceeds that limit.
> +               push(@tar_p_flags, "--owner=3D0", "--group=3D0");

Interesting.  BSD tar accepts those too, so here's an update to my
previous patch.

> While this fixes the test, I am now not sure what the broader implication=
s are
> for --format=3Dustar for pg_waldump in the broader discussion?

I think users who have their own tar scripts will mostly be
unaffected, but a small minority will see the new error if they try to
use pg_verifybackup or pg_waldump, and they'll find their way to
--format=3Dustar, and then they might see the UID/GID error in their own
.tar production scripts, and find their way to adding those switches
too.  Seems OK?  Especially with a documentation note once we've
settle all of this.

--0000000000005066bb064e831d63
Content-Type: text/x-patch; charset="US-ASCII";
	name="v2-0001-Improve-tar-portability-logic-from-ebba64c0.patch"
Content-Disposition: attachment;
	filename="v2-0001-Improve-tar-portability-logic-from-ebba64c0.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_mni597y70>
X-Attachment-Id: f_mni597y70

RnJvbSBhMmMwNjVmZDVjZWU1M2JjZGUyY2NhZjkyOTM5NzM3ZThmYzA3ZjA2IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBUaG9tYXMgTXVucm8gPHRob21hcy5tdW5yb0BnbWFpbC5jb20+
CkRhdGU6IEZyaSwgMyBBcHIgMjAyNiAxMjowMzo1NiArMTMwMApTdWJqZWN0OiBbUEFUQ0ggdjJd
IEltcHJvdmUgdGFyIHBvcnRhYmlsaXR5IGxvZ2ljIGZyb20gZWJiYTY0YzAuCgoqIEdOVSBhbmQg
QlNEIHRhciBib3RoIHVuZGVyc3RhbmQgLS1mb3JtYXQ9dXN0YXIuCiogV2luZG93cyBsYWNrcyAv
ZGV2L251bGwsIGJ1dCBwZXJsIGtub3dzIGl0cyBsb2NhbCBuYW1lLgoqIHVzdGFyIGZvcm1hdCBk
b2Vzbid0IGxpa2UgbGFyZ2UgVUlEL0dJRCB2YWx1ZXMsIHNvIHNldCB0aGVtIHRvIDAuCgpCYWNr
cGF0Y2gtdGhyb3VnaDogMTgKQ28tYXV0aG9yZWQtYnk6IFRob21hcyBNdW5ybyA8dGhvbWFzLm11
bnJvQGdtYWlsLmNvbT4KQ28tYXV0aG9yZWQtYnk6IFNhbWkgSW1zZWloIDxzYW1pbXNlaWhAZ21h
aWwuY29tPgpEaXNjdXNzaW9uOiBodHRwczovL3Bvc3Rnci5lcy9tLzM2NzYyMjkuMTc3NTE3MDI1
MCU0MHNzcy5wZ2gucGEudXMKRGlzY3Vzc2lvbjogaHR0cHM6Ly9wb3N0Z3IuZXMvbS9DQUE1Ulow
dHQ4OU1nTmk0LTBGNG9uSCUyQi1URlNzeXNGak1NLXRCYzZhWGJ1UXY1eEJYdyU0MG1haWwuZ21h
aWwuY29tCi0tLQogc3JjL3Rlc3QvcGVybC9Qb3N0Z3JlU1FML1Rlc3QvVXRpbHMucG0gfCAyMCAr
KysrKysrKy0tLS0tLS0tLS0tLQogMSBmaWxlIGNoYW5nZWQsIDggaW5zZXJ0aW9ucygrKSwgMTIg
ZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvc3JjL3Rlc3QvcGVybC9Qb3N0Z3JlU1FML1Rlc3Qv
VXRpbHMucG0gYi9zcmMvdGVzdC9wZXJsL1Bvc3RncmVTUUwvVGVzdC9VdGlscy5wbQppbmRleCAx
MjA5OTlmNmFjOS4uMzcwYWNmY2VmN2UgMTAwNjQ0Ci0tLSBhL3NyYy90ZXN0L3BlcmwvUG9zdGdy
ZVNRTC9UZXN0L1V0aWxzLnBtCisrKyBiL3NyYy90ZXN0L3BlcmwvUG9zdGdyZVNRTC9UZXN0L1V0
aWxzLnBtCkBAIC0xMzI4LDIxICsxMzI4LDE3IEBAIHN1YiB0YXJfcG9ydGFiaWxpdHlfb3B0aW9u
cwogCiAJIyBHTlUgdGFyIHR5cGljYWxseSBwcm9kdWNlcyBnbnUtZm9ybWF0IGFyY2hpdmVzLCB3
aGljaCB3ZSBjYW4gcmVhZCBmaW5lLgogCSMgQnV0IHNvbWUgcGxhdGZvcm1zIGNvbmZpZ3VyZSBp
dCB0byBkZWZhdWx0IHRvIHBvc2l4L3BheCBmb3JtYXQsIGFuZAotCSMgYXBwYXJlbnRseSB0aGV5
IGVuYWJsZSAtLXNwYXJzZSB0b28uICBPdmVycmlkZSB0aGF0LgotCWlmIChzeXN0ZW0oIiR0YXIg
LS1mb3JtYXQ9dXN0YXIgLWMgLU8gL2Rldi9udWxsID4vZGV2L251bGwgMj4vZGV2L251bGwiKQot
CQk9PSAwKQotCXsKLQkJcHVzaChAdGFyX3BfZmxhZ3MsICItLWZvcm1hdD11c3RhciIpOwotCX0K
LQotCSMgYnNkdGFyIGFsc28gYXJjaGl2ZXMgc3BhcnNlIGZpbGVzIGJ5IGRlZmF1bHQsIGJ1dCBp
dCBzcGVsbHMgdGhlIHN3aXRjaAotCSMgdG8gZGlzYWJsZSB0aGF0IGRpZmZlcmVudGx5LgotCWlm
IChzeXN0ZW0oIiR0YXIgLS1uby1yZWFkLXNwYXJzZSAtYyAtIC9kZXYvbnVsbCA+L2Rldi9udWxs
IDI+L2Rldi9udWxsIikKKwkjIGFwcGFyZW50bHkgdGhleSBlbmFibGUgLS1zcGFyc2UgdG9vLiAg
QlNEIHRhciBkb2VzIHNvbWV0aGluZyBzaW1pbGFyLgorCSMKKwkjIHVzdGFyIGZvcm1hdCBzdXBw
b3J0cyBVSURzIG9ubHkgdXAgdG8gMl4yMSAoMjA5NzE1MSkuICBPdmVycmlkZQorCSMgb3duZXIv
Z3JvdXAgdG8gYXZvaWQgZmFpbHVyZXMgb24gc3lzdGVtcyB3aGVyZSB0aGUgcnVubmluZyB1c2Vy
J3MgVUlEL0dJRAorCSMgZXhjZWVkcyB0aGF0IGxpbWl0LgorCW15ICRkZXZudWxsID0gRmlsZTo6
U3BlYy0+ZGV2bnVsbCgpOworCWlmIChzeXN0ZW0oIiR0YXIgLS1mb3JtYXQ9dXN0YXIgLS1vd25l
cj0wIC0tZ3JvdXA9MCAtYyAkZGV2bnVsbCA+JGRldm51bGwgMj4kZGV2bnVsbCIpCiAJCT09IDAp
CiAJewotCQlwdXNoKEB0YXJfcF9mbGFncywgIi0tbm8tcmVhZC1zcGFyc2UiKTsKKwkJcHVzaChA
dGFyX3BfZmxhZ3MsICItLWZvcm1hdD11c3RhciIsICItLW93bmVyPTAiLCAiLS1ncm91cD0wIik7
CiAJfQotCiAJcmV0dXJuIEB0YXJfcF9mbGFnczsKIH0KIAotLSAKMi41My4wCgo=
--0000000000005066bb064e831d63--