From: Tom Lane <tgl@sss.pgh.pa.us>
To: Thomas Munro <thomas.munro@gmail.com>
cc: Andres Freund <andres@anarazel.de>, Michael Paquier <michael@paquier.xyz>,
        Andrew Dunstan <andrew@dunslane.net>, Amul Sul <sulamul@gmail.com>,
        Zsolt Parragi <zsolt.parragi@percona.com>,
        Robert Haas <robertmhaas@gmail.com>, Chao Li <li.evan.chao@gmail.com>,
        Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>,
        Fujii Masao <masao.fujii@gmail.com>,
        Jakub Wartak <jakub.wartak@enterprisedb.com>,
        PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
In-reply-to: 
 <CA+hUKGL2dppjO4o28ZY7n_LTWviKLAi-7KZ=tx5w2HGevCEYPA@mail.gmail.com>
References: <2250061.1774104346@sss.pgh.pa.us>
 <2555285.1774131847@sss.pgh.pa.us> <2609460.1774153487@sss.pgh.pa.us>
 <CAD5tBcLVWKnph3iB-VPuPKR0dCckOJRFZW2-4H7HTTmhw8-vOg@mail.gmail.com>
 <2790913.1774200584@sss.pgh.pa.us> <2880042.1774203473@sss.pgh!!.pa.us>
 <CAD5tBcLsYDz+Nzx8MryjxiKaN3fGKd4ZgXuN1Jn=CYxw9dh+AA@mail.gmail.com>
 <3341199.1774221191@sss.pgh.pa.us> <3424809.1774234940@sss.pgh.pa.us>
 <acIA8KKBTv-T53YG@paquier.xyz>
 <x2tknjejjouleunkqrvpnwn2tuulunybinycidefm3wmnsyhht@pw5uo3wrqx43>
 <CA+hUKGL2dppjO4o28ZY7n_LTWviKLAi-7KZ=tx5w2HGevCEYPA@mail.gmail.com>
Comments: In-reply-to Thomas Munro <thomas.munro@gmail.com>
	message dated "Sun, 29 Mar 2026 10:36:06 +1300"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1624715.1774736283.1@sss.pgh.pa.us>
Date: Sat, 28 Mar 2026 18:18:03 -0400
Message-ID: <1624716.1774736283@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/1624716.1774736283%40sss.pgh.pa.us>
Precedence: bulk

Thomas Munro <thomas.munro@gmail.com> writes:
> It seems like the place we'd be looking for the file is in
> astreamer_tar_header(), so I added in some caveman debugging:

> XXXX [PaxHeader/000000010000000000000002] XXXX
> XXXX [GNUSparseFile.0/000000010000000000000002] XXXX

After a bit of research, I see the problem: your tar has decided to
store 000000010000000000000002 as a sparse file, and evidently it
inserts "GNUSparseFile.0/" into the file name when it does that.
Our tar-parsing logic will not recognize that name as being one
of the WAL files it wants; and even if it did, it doesn't know
how to de-sparse the file contents, so it would get the wrong
output.

I'm pretty disinclined to try to support sparse files here,
especially with ~10 days till feature freeze.

However ... I do not find any indication in the GNU tar docs
that it produces sparse files by default.  It looks like you
need to say -S/--sparse to make that happen.  Maybe you have
a version that's been hacked to make that the default?

			regards, tom lane