From: Tom Lane <tgl@sss.pgh.pa.us>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: Amul Sul <sulamul@gmail.com>
Cc: Zsolt Parragi <zsolt.parragi@percona.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Chao Li <li.evan.chao@gmail.com>
Cc: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Cc: Fujii Masao <masao.fujii@gmail.com>
Cc: Jakub Wartak <jakub.wartak@enterprisedb.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
Date: Sat, 28 Mar 2026 18:18:03 -0400
Message-ID: <1624716.1774736283@sss.pgh.pa.us> (raw)
In-Reply-To: <CA+hUKGL2dppjO4o28ZY7n_LTWviKLAi-7KZ=tx5w2HGevCEYPA@mail.gmail.com>
References: <2250061.1774104346@sss.pgh.pa.us>
<2555285.1774131847@sss.pgh.pa.us>
<2609460.1774153487@sss.pgh.pa.us>
<CAD5tBcLVWKnph3iB-VPuPKR0dCckOJRFZW2-4H7HTTmhw8-vOg@mail.gmail.com>
<2790913.1774200584@sss.pgh.pa.us>
<2880042.1774203473@sss.pgh!!.pa.us>
<CAD5tBcLsYDz+Nzx8MryjxiKaN3fGKd4ZgXuN1Jn=CYxw9dh+AA@mail.gmail.com>
<3341199.1774221191@sss.pgh.pa.us>
<3424809.1774234940@sss.pgh.pa.us>
<acIA8KKBTv-T53YG@paquier.xyz>
<x2tknjejjouleunkqrvpnwn2tuulunybinycidefm3wmnsyhht@pw5uo3wrqx43>
<CA+hUKGL2dppjO4o28ZY7n_LTWviKLAi-7KZ=tx5w2HGevCEYPA@mail.gmail.com>
Thomas Munro <thomas.munro@gmail.com> writes:
> It seems like the place we'd be looking for the file is in
> astreamer_tar_header(), so I added in some caveman debugging:
> XXXX [PaxHeader/000000010000000000000002] XXXX
> XXXX [GNUSparseFile.0/000000010000000000000002] XXXX
After a bit of research, I see the problem: your tar has decided to
store 000000010000000000000002 as a sparse file, and evidently it
inserts "GNUSparseFile.0/" into the file name when it does that.
Our tar-parsing logic will not recognize that name as being one
of the WAL files it wants; and even if it did, it doesn't know
how to de-sparse the file contents, so it would get the wrong
output.
I'm pretty disinclined to try to support sparse files here,
especially with ~10 days till feature freeze.
However ... I do not find any indication in the GNU tar docs
that it produces sparse files by default. It looks like you
need to say -S/--sparse to make that happen. Maybe you have
a version that's been hacked to make that the default?
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tgl@sss.pgh.pa.us, thomas.munro@gmail.com, andres@anarazel.de, michael@paquier.xyz, andrew@dunslane.net, sulamul@gmail.com, zsolt.parragi@percona.com, robertmhaas@gmail.com, li.evan.chao@gmail.com, anthonin.bonnefoy@datadoghq.com, masao.fujii@gmail.com, jakub.wartak@enterprisedb.com, pgsql-hackers@lists.postgresql.org
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
In-Reply-To: <1624716.1774736283@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox