MIME-Version: 1.0
References: <2250061.1774104346@sss.pgh.pa.us>
 <2555285.1774131847@sss.pgh.pa.us>
 <2609460.1774153487@sss.pgh.pa.us>
 <CAD5tBcLVWKnph3iB-VPuPKR0dCckOJRFZW2-4H7HTTmhw8-vOg@mail.gmail.com>
 <2790913.1774200584@sss.pgh.pa.us> <2880042.1774203473@sss.pgh!!.pa.us>
 <CAD5tBcLsYDz+Nzx8MryjxiKaN3fGKd4ZgXuN1Jn=CYxw9dh+AA@mail.gmail.com>
 <3341199.1774221191@sss.pgh.pa.us> <3424809.1774234940@sss.pgh.pa.us>
 <acIA8KKBTv-T53YG@paquier.xyz>
 <x2tknjejjouleunkqrvpnwn2tuulunybinycidefm3wmnsyhht@pw5uo3wrqx43>
 <CA+hUKGL2dppjO4o28ZY7n_LTWviKLAi-7KZ=tx5w2HGevCEYPA@mail.gmail.com>
 <1624716.1774736283@sss.pgh.pa.us> <1626907.1774737417@sss.pgh.pa.us>
 <97a382c0-1f19-4ea0-951f-e37e6abc34a3@vondra.me>
 <1630755.1774739531@sss.pgh.pa.us>
 <c3a2b730-9d90-4a66-ae5e-50f6d77922d6@vondra.me>
 <CA+hUKGJyvdyWMC-RW1njqevD-q_gTbFq+DyDiFpUJVaG+DY20w@mail.gmail.com>
 <1873141.1774823011@sss.pgh.pa.us>
 <CA+hUKG+Pqz5=YQG_=8ho0YsTfn2HWOsJQWqS4j0q8QQWweJP9w@mail.gmail.com>
 <3049460.1775067940@sss.pgh.pa.us>
In-Reply-To: <3049460.1775067940@sss.pgh.pa.us>
From: Thomas Munro <thomas.munro@gmail.com>
Date: Thu, 2 Apr 2026 13:16:17 +1300
Message-ID: 
 <CA+hUKGJv2hpjMDwG31JSR7TfFUxNPPtQKMm6mHVVRMAT0XppBQ@mail.gmail.com>
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Tomas Vondra <tomas@vondra.me>, Andres Freund <andres@anarazel.de>,
	Michael Paquier <michael@paquier.xyz>, Andrew Dunstan <andrew@dunslane.net>,
 Amul Sul <sulamul@gmail.com>,
	Zsolt Parragi <zsolt.parragi@percona.com>,
 Robert Haas <robertmhaas@gmail.com>,
	Chao Li <li.evan.chao@gmail.com>,
	Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>,
 Fujii Masao <masao.fujii@gmail.com>,
	Jakub Wartak <jakub.wartak@enterprisedb.com>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: multipart/mixed; boundary="0000000000003ed715064e6f1e0a"
Archived-At: 
 <https://www.postgresql.org/message-id/CA%2BhUKGJv2hpjMDwG31JSR7TfFUxNPPtQKMm6mHVVRMAT0XppBQ%40mail.gmail.com>
Precedence: bulk

--0000000000003ed715064e6f1e0a
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Apr 2, 2026 at 7:25=E2=80=AFAM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Also, if we are admitting the possibility that what we are reading
> was made by a platform-supplied tar and not our own code, I think
> it verges on lunacy to behave as though unsupported typeflags are
> regular files.

Yeah, if this is the first time we parse files we didn't make then
that makes total sense.  I was a bit unsure of that question when I
suggested we reject pax only after we've failed to find a file, in
case there are scenarios that work today with harmless ignorable pax
headers that don't change the file name.

> So I think we need something more or less like the attached.

LGTM.  Tested with both tars here.  I updated that little test patch
for this.  Not sure if you think it's worth a test though, now that
it's so simple.

@Andrew: I tried using File::Spec->devnull() this time.  Are you able
to check if this works OK on Windows, applied on top of Tom's patch?
AFAIK should be able to run this new test and pass, not skip it.  But
it could be that the shell invocation needs tweaking.  It's hard to
tell from CI.  (Huh, apparently Windows ships a copy of BSD tar as
C:\Windows\System32\tar.exe these days.)

--0000000000003ed715064e6f1e0a
Content-Type: text/x-patch; charset="US-ASCII";
	name="0001-Test-rejection-of-pax-extended-tar-files.patch"
Content-Disposition: attachment;
	filename="0001-Test-rejection-of-pax-extended-tar-files.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_mngptctv0>
X-Attachment-Id: f_mngptctv0

RnJvbSA1N2I5ZDFhNThhOWViMzBmZWVjNzJiMGNjY2U2YmRmZDVlMWVmOTJlIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBUaG9tYXMgTXVucm8gPHRob21hcy5tdW5yb0BnbWFpbC5jb20+
CkRhdGU6IE1vbiwgMzAgTWFyIDIwMjYgMTg6MjA6MDkgKzEzMDAKU3ViamVjdDogW1BBVENIXSBU
ZXN0IHJlamVjdGlvbiBvZiBwYXggZXh0ZW5kZWQgdGFyIGZpbGVzLgoKQWxzbyBjaGFuZ2UgODUy
ZGU1NzkncyBzaGVsbCBpbnZvY2F0aW9uIHRvIHVzZSBOVUwgb24gV2luZG93cyBpbnN0ZWFkIG9m
Ci9kZXYvbnVsbCwgcGVyIGZlZWRiYWNrIGZyb20gQW5kcmV3IER1bnN0YW4uICBUaGF0IGNvbW1h
bmQgd2Fzbid0CmV4cGVjdGVkIHRvIHN1Y2NlZWQgb24gV2luZG93cywgYnV0IHRoZSBzaW1pbGFy
IHVzYWdlIGFkZGVkIGhlcmUgbWlnaHQuCi0tLQogc3JjL2Jpbi9wZ193YWxkdW1wL3QvMDAxX2Jh
c2ljLnBsIHwgNDIgKysrKysrKysrKysrKysrKysrKysrKysrKy0tLS0tLQogMSBmaWxlIGNoYW5n
ZWQsIDM1IGluc2VydGlvbnMoKyksIDcgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvc3JjL2Jp
bi9wZ193YWxkdW1wL3QvMDAxX2Jhc2ljLnBsIGIvc3JjL2Jpbi9wZ193YWxkdW1wL3QvMDAxX2Jh
c2ljLnBsCmluZGV4IGNlMWY2YWEzMGMwLi4zYjk5YThmNmQ0ZiAxMDA2NDQKLS0tIGEvc3JjL2Jp
bi9wZ193YWxkdW1wL3QvMDAxX2Jhc2ljLnBsCisrKyBiL3NyYy9iaW4vcGdfd2FsZHVtcC90LzAw
MV9iYXNpYy5wbApAQCAtNSw2ICs1LDcgQEAgdXNlIHN0cmljdDsKIHVzZSB3YXJuaW5ncyBGQVRB
TCA9PiAnYWxsJzsKIHVzZSBDd2Q7CiB1c2UgRmlsZTo6Q29weTsKK3VzZSBGaWxlOjpTcGVjOwog
dXNlIFBvc3RncmVTUUw6OlRlc3Q6OkNsdXN0ZXI7CiB1c2UgUG9zdGdyZVNRTDo6VGVzdDo6VXRp
bHM7CiB1c2UgVGVzdDo6TW9yZTsKQEAgLTEzLDEyICsxNCwxNCBAQCB1c2UgTGlzdDo6VXRpbCBx
dyhzaHVmZmxlKTsKIG15ICR0YXIgPSAkRU5We1RBUn07CiBteSBAdGFyX2NfZmxhZ3M7CiAKLSMg
QnkgZGVmYXVsdCwgYnNkdGFyIGFyY2hpdmVzIHNwYXJzZSBmaWxlcyBpbiBHTlUgdGFyJ3MgLS1m
b3JtYXQ9cG9zaXggLS1zcGFyc2UKLSMgZm9ybWF0LCBzbyBwZ193YWxkdW1wIGNhbid0IGZpbmQg
ZmlsZXMgdGhhdCBaRlMgaGFzIGRlY2lkZWQgdG8gc3RvcmUgd2l0aAotIyBob2xlcy4gIFR1cm4g
dGhhdCBvZmYuCi1pZiAoc3lzdGVtKCIkdGFyIC0tbm8tcmVhZC1zcGFyc2UgLWMgLSAvZGV2L251
bGwgPiAvZGV2L251bGwiKSA9PSAwKQorbXkgJGRldm51bGwgPSBGaWxlOjpTcGVjLT5kZXZudWxs
KCk7CisKKyMgQnkgZGVmYXVsdCwgYnNkdGFyIGFyY2hpdmVzIHNwYXJzZSBmaWxlcyBpbiBHTlUg
dGFyJ3MgLS1mb3JtYXQ9cGF4IC0tc3BhcnNlCisjIGZvcm1hdCwgc28gcGdfd2FsZHVtcCByZWpl
Y3RzIFdBTCB0aGF0IFpGUyBoYXMgZGVjaWRlZCB0byBzdG9yZSB3aXRoIGhvbGVzLgorIyBUdXJu
IHRoYXQgb2ZmLgoraWYgKHN5c3RlbSgiJHRhciAtLW5vLXJlYWQtc3BhcnNlIC1jICRkZXZudWxs
ID4gJGRldm51bGwiKSA9PSAwKQogewotICBwdXNoKEB0YXJfY19mbGFncywgIi0tbm8tcmVhZC1z
cGFyc2UiKTsKKwlwdXNoKEB0YXJfY19mbGFncywgIi0tbm8tcmVhZC1zcGFyc2UiKTsKIH0KIAog
cHJvZ3JhbV9oZWxwX29rKCdwZ193YWxkdW1wJyk7CkBAIC0zMzksNyArMzQyLDcgQEAgc3ViIHRl
c3RfcGdfd2FsZHVtcAogIyBDcmVhdGUgYSB0YXIgYXJjaGl2ZSwgc2h1ZmZsZSB0aGUgZmlsZSBv
cmRlcgogc3ViIGdlbmVyYXRlX2FyY2hpdmUKIHsKLQlteSAoJGFyY2hpdmUsICRkaXJlY3Rvcnks
ICRjb21wcmVzc2lvbl9mbGFncykgPSBAXzsKKwlteSAoJGFyY2hpdmUsICRkaXJlY3RvcnksICRj
b21wcmVzc2lvbl9mbGFncywgQGV4dHJhX2ZsYWdzKSA9IEBfOwogCiAJbXkgQGZpbGVzOwogCW9w
ZW5kaXIgbXkgJGRoLCAkZGlyZWN0b3J5IG9yIGRpZSAib3BlbmRpcjogJCEiOwpAQCAtMzU1LDcg
KzM1OCw3IEBAIHN1YiBnZW5lcmF0ZV9hcmNoaXZlCiAJIyBtb3ZlIGludG8gdGhlIFdBTCBkaXJl
Y3RvcnkgYmVmb3JlIGFyY2hpdmluZyBmaWxlcwogCW15ICRjd2QgPSBnZXRjd2Q7CiAJY2hkaXIo
JGRpcmVjdG9yeSkgfHwgZGllICJjaGRpcjogJCEiOwotCWNvbW1hbmRfb2soWyR0YXIsIEB0YXJf
Y19mbGFncywgJGNvbXByZXNzaW9uX2ZsYWdzLCAkYXJjaGl2ZSwgQGZpbGVzXSk7CisJY29tbWFu
ZF9vayhbJHRhciwgQGV4dHJhX2ZsYWdzLCBAdGFyX2NfZmxhZ3MsICRjb21wcmVzc2lvbl9mbGFn
cywgJGFyY2hpdmUsIEBmaWxlc10pOwogCWNoZGlyKCRjd2QpIHx8IGRpZSAiY2hkaXI6ICQhIjsK
IH0KIApAQCAtNDc3LDQgKzQ4MCwyOSBAQCBmb3IgbXkgJHNjZW5hcmlvIChAc2NlbmFyaW9zKQog
CX0KIH0KIAorU0tJUDoKK3sKKwlza2lwICJ0YXIgY29tbWFuZCBpcyBub3QgYXZhaWxhYmxlIiwg
MQorCQlpZiAhZGVmaW5lZCAkdGFyOworCisJc2tpcCAidGFyIGNvbW1hbmQgZG9lc24ndCB1bmRl
cnN0YW5kIC0tZm9ybWF0PXBheCIsIDEKKwkJaWYgc3lzdGVtKCIkdGFyIC0tZm9ybWF0PXBheCAt
YyAiIC4KKwkJCQkgICRub2RlLT5kYXRhX2RpciAuICIvcGdfd2FsLyogPiAkZGV2bnVsbCIpICE9
IDA7CisKKwlnZW5lcmF0ZV9hcmNoaXZlKCR0bXBfZGlyIC4gJy9wZ193YWxfcGF4LnRhcicsCisJ
CQkJCSAkbm9kZS0+ZGF0YV9kaXIgLiAnL3BnX3dhbCcsCisJCQkJCSAnLWNmJywKKwkJCQkJICgi
LS1mb3JtYXQ9cGF4IikpOworCisJY29tbWFuZF9mYWlsc19saWtlKAorCQlbCisJCQkncGdfd2Fs
ZHVtcCcsCisJCQknLS1wYXRoJyA9PiAkdG1wX2RpciAuICcvcGdfd2FsX3BheC50YXInLAorCQkJ
Jy0tc3RhcnQnID0+ICRzdGFydF9sc24sCisJCQknLS1lbmQnID0+ICRlbmRfbHNuLAorCQldLAor
CQlxci9lcnJvcjogcGF4IGV4dGVuc2lvbnMgdG8gdGFyIGZvcm1hdCBhcmUgbm90IHN1cHBvcnRl
ZC8sCisJCSdmYWlscyBpZiBwYXggZXh0ZW5kZWQgaGVhZGVyIGlzIGRldGVjdGVkJyk7Cit9CisK
IGRvbmVfdGVzdGluZygpOwotLSAKMi41Mi4wCgo=
--0000000000003ed715064e6f1e0a--