MIME-Version: 1.0
References: <2250061.1774104346@sss.pgh.pa.us>
 <2555285.1774131847@sss.pgh.pa.us>
 <2609460.1774153487@sss.pgh.pa.us>
 <CAD5tBcLVWKnph3iB-VPuPKR0dCckOJRFZW2-4H7HTTmhw8-vOg@mail.gmail.com>
 <2790913.1774200584@sss.pgh.pa.us> <2880042.1774203473@sss.pgh!!.pa.us>
 <CAD5tBcLsYDz+Nzx8MryjxiKaN3fGKd4ZgXuN1Jn=CYxw9dh+AA@mail.gmail.com>
 <3341199.1774221191@sss.pgh.pa.us> <3424809.1774234940@sss.pgh.pa.us>
 <acIA8KKBTv-T53YG@paquier.xyz>
 <x2tknjejjouleunkqrvpnwn2tuulunybinycidefm3wmnsyhht@pw5uo3wrqx43>
 <CA+hUKGL2dppjO4o28ZY7n_LTWviKLAi-7KZ=tx5w2HGevCEYPA@mail.gmail.com>
 <1624716.1774736283@sss.pgh.pa.us> <1626907.1774737417@sss.pgh.pa.us>
In-Reply-To: <1626907.1774737417@sss.pgh.pa.us>
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sun, 29 Mar 2026 12:15:39 +1300
Message-ID: 
 <CA+hUKGK-GaHRh5mO0WKWO6MiL-iJ=+j_vVg4f-3pQXATNfRC2Q@mail.gmail.com>
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Tomas Vondra <tomas@vondra.me>, Andres Freund <andres@anarazel.de>,
	Michael Paquier <michael@paquier.xyz>, Andrew Dunstan <andrew@dunslane.net>,
 Amul Sul <sulamul@gmail.com>,
	Zsolt Parragi <zsolt.parragi@percona.com>,
 Robert Haas <robertmhaas@gmail.com>,
	Chao Li <li.evan.chao@gmail.com>,
	Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>,
 Fujii Masao <masao.fujii@gmail.com>,
	Jakub Wartak <jakub.wartak@enterprisedb.com>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: multipart/mixed; boundary="000000000000fd5c81064e1dcdd7"
Archived-At: 
 <https://www.postgresql.org/message-id/CA%2BhUKGK-GaHRh5mO0WKWO6MiL-iJ%3D%2Bj_vVg4f-3pQXATNfRC2Q%40mail.gmail.com>
Precedence: bulk

--000000000000fd5c81064e1dcdd7
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, Mar 29, 2026 at 11:37=E2=80=AFAM Tom Lane <tgl@sss.pgh.pa.us> wrote=
:
> I wrote:
> > However ... I do not find any indication in the GNU tar docs
> > that it produces sparse files by default.  It looks like you
> > need to say -S/--sparse to make that happen.  Maybe you have
> > a version that's been hacked to make that the default?
>
> Bleah.  Digging in the man pages at freebsd.org, I read
>
>     --read-sparse
>                (c, r, u modes only) Read sparse file  information  from  =
disk.
>                This  is the reverse of --no-read-sparse and the default b=
ehav-
>                ior.
>
> It's apparently been there and been default since FreeBSD 13.1.
> This leads one to wonder how come BF member dikkop is managing
> to run this test successfully.  I speculate that it's using a
> filesystem type that doesn't do sparse files (cc'ing Vondra
> for confirmation on that).
>
> It looks like to make this test stable on modern FreeBSD,
> we need to see if tar accepts --no-read-sparse and use that
> switch if so.

Yeah.  Here's my attempt at perl.

I think your Mac probably has a similar tar program BTW... but apfs
probably doesn't go around making holes visible to lseek()
automatically or at least as eagerly as my ZFS system.

--000000000000fd5c81064e1dcdd7
Content-Type: text/x-patch; charset="US-ASCII";
	name="0001-Fix-pg_waldump-test-for-libarchive-tar.patch"
Content-Disposition: attachment;
	filename="0001-Fix-pg_waldump-test-for-libarchive-tar.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_mnaxxp7j0>
X-Attachment-Id: f_mnaxxp7j0

RnJvbSBiMWE5MTMxZDA3MmRhZGZlOWM1NjY2ZTc1YTkzMmJhNzBiOGQyZjk5IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBUaG9tYXMgTXVucm8gPHRob21hcy5tdW5yb0BnbWFpbC5jb20+
CkRhdGU6IFN1biwgMjkgTWFyIDIwMjYgMTI6MDM6NDIgKzEzMDAKU3ViamVjdDogW1BBVENIXSBG
aXggcGdfd2FsZHVtcCB0ZXN0IGZvciBsaWJhcmNoaXZlIHRhci4KCmxpYmFyY2hpdmUgdGFyICh0
aGUgb25lIHNoaXBwZWQgb24gbWFjT1MsICpCU0Qgc3lzdGVtcykgbWlnaHQgZGVjaWRlIHRvCmFy
Y2hpdmUgbm9uLXN0YW5kYXJkIHNwYXJzZSBlbmNvZGluZyB3aXRoIEdOVSBleHRlbnNpb25zICh1
bmxpa2UgR05VIHRhcgppdHNlbGYpIGJ5IGRlZmF1bHQuICBwZ193YWxkdW1wIGNhbid0IHJlYWQg
dGhlbS4gIFN1cHByZXNzIHRoYXQsIGlmICRUQVIKdW5kZXJzdGFuZHMgLS1uby1zcGFyc2UtZmls
ZXMuCgpEaXNjdXNzaW9uOiBodHRwczovL3Bvc3Rnci5lcy9tLzE2MjQ3MTYuMTc3NDczNjI4MyU0
MHNzcy5wZ2gucGEudXMKLS0tCiBzcmMvYmluL3BnX3dhbGR1bXAvdC8wMDFfYmFzaWMucGwgfCAx
MSArKysrKysrKystLQogMSBmaWxlIGNoYW5nZWQsIDkgaW5zZXJ0aW9ucygrKSwgMiBkZWxldGlv
bnMoLSkKCmRpZmYgLS1naXQgYS9zcmMvYmluL3BnX3dhbGR1bXAvdC8wMDFfYmFzaWMucGwgYi9z
cmMvYmluL3BnX3dhbGR1bXAvdC8wMDFfYmFzaWMucGwKaW5kZXggOGJiOGZhMjI1ZjYuLmQ5MTEy
OTZiYjY2IDEwMDY0NAotLS0gYS9zcmMvYmluL3BnX3dhbGR1bXAvdC8wMDFfYmFzaWMucGwKKysr
IGIvc3JjL2Jpbi9wZ193YWxkdW1wL3QvMDAxX2Jhc2ljLnBsCkBAIC0xMSw2ICsxMSwxNCBAQCB1
c2UgVGVzdDo6TW9yZTsKIHVzZSBMaXN0OjpVdGlsIHF3KHNodWZmbGUpOwogCiBteSAkdGFyID0g
JEVOVntUQVJ9OworbXkgQFRBUl9DX0ZMQUdTOworCisjIGxpYmFyY2hpdmUgdGFyIChhcyBmb3Vu
ZCBvbiAqQlNEIGFuZCBtYWNPUykgbWlnaHQgY3JlYXRlIHNwYXJzZSBmaWxlcyBieQorIyBkZWZh
dWx0LCBhbmQgd2UgY2FuJ3QgcmVhZCB0aGVtCitpZiAoc3lzdGVtKCIkdGFyIC0tbm8tcmVhZC1z
cGFyc2UgLWMgLSAvZGV2L251bGwgPiAvZGV2L251bGwiKSA9PSAwKQoreworICBwdXNoKEBUQVJf
Q19GTEFHUywgIi0tbm8tcmVhZC1zcGFyc2UiKTsKK30KIAogcHJvZ3JhbV9oZWxwX29rKCdwZ193
YWxkdW1wJyk7CiBwcm9ncmFtX3ZlcnNpb25fb2soJ3BnX3dhbGR1bXAnKTsKQEAgLTMzMSw3ICsz
MzksNiBAQCBzdWIgdGVzdF9wZ193YWxkdW1wCiBzdWIgZ2VuZXJhdGVfYXJjaGl2ZQogewogCW15
ICgkYXJjaGl2ZSwgJGRpcmVjdG9yeSwgJGNvbXByZXNzaW9uX2ZsYWdzKSA9IEBfOwotCiAJbXkg
QGZpbGVzOwogCW9wZW5kaXIgbXkgJGRoLCAkZGlyZWN0b3J5IG9yIGRpZSAib3BlbmRpcjogJCEi
OwogCXdoaWxlIChteSAkZW50cnkgPSByZWFkZGlyICRkaCkgewpAQCAtMzQ2LDcgKzM1Myw3IEBA
IHN1YiBnZW5lcmF0ZV9hcmNoaXZlCiAJIyBtb3ZlIGludG8gdGhlIFdBTCBkaXJlY3RvcnkgYmVm
b3JlIGFyY2hpdmluZyBmaWxlcwogCW15ICRjd2QgPSBnZXRjd2Q7CiAJY2hkaXIoJGRpcmVjdG9y
eSkgfHwgZGllICJjaGRpcjogJCEiOwotCWNvbW1hbmRfb2soWyR0YXIsICRjb21wcmVzc2lvbl9m
bGFncywgJGFyY2hpdmUsIEBmaWxlc10pOworCWNvbW1hbmRfb2soWyR0YXIsIEBUQVJfQ19GTEFH
UywgJGNvbXByZXNzaW9uX2ZsYWdzLCAkYXJjaGl2ZSwgQGZpbGVzXSk7CiAJY2hkaXIoJGN3ZCkg
fHwgZGllICJjaGRpcjogJCEiOwogfQogCi0tIAoyLjUyLjAKCg==
--000000000000fd5c81064e1dcdd7--