Message-ID: <0771308b-03f4-455a-9748-a2e0cd4aab03@dunslane.net>
Date: Wed, 1 Apr 2026 10:19:32 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
To: Andres Freund <andres@anarazel.de>
Cc: Thomas Munro <thomas.munro@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>,
 Tomas Vondra <tomas@vondra.me>, Michael Paquier <michael@paquier.xyz>,
 Amul Sul <sulamul@gmail.com>, Zsolt Parragi <zsolt.parragi@percona.com>,
 Robert Haas <robertmhaas@gmail.com>, Chao Li <li.evan.chao@gmail.com>,
 Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>,
 Fujii Masao <masao.fujii@gmail.com>,
 Jakub Wartak <jakub.wartak@enterprisedb.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: 
 <CA+hUKGL2dppjO4o28ZY7n_LTWviKLAi-7KZ=tx5w2HGevCEYPA@mail.gmail.com>
 <1624716.1774736283@sss.pgh.pa.us> <1626907.1774737417@sss.pgh.pa.us>
 <97a382c0-1f19-4ea0-951f-e37e6abc34a3@vondra.me>
 <1630755.1774739531@sss.pgh.pa.us>
 <c3a2b730-9d90-4a66-ae5e-50f6d77922d6@vondra.me>
 <CA+hUKGJyvdyWMC-RW1njqevD-q_gTbFq+DyDiFpUJVaG+DY20w@mail.gmail.com>
 <1873141.1774823011@sss.pgh.pa.us>
 <CA+hUKG+Pqz5=YQG_=8ho0YsTfn2HWOsJQWqS4j0q8QQWweJP9w@mail.gmail.com>
 <7c394909-7680-4305-885e-8c7f18ed4441@dunslane.net>
 <e4twyomp2vc2trbhqa4uszjobyeccctid4m3fzuzokxc5qdvri@mkghlmmobjgn>
From: Andrew Dunstan <andrew@dunslane.net>
Content-Language: en-US
Autocrypt: addr=andrew@dunslane.net; keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNJEFuZHJldyBEdW5zdGFuIDxhbmRyZXdAZHVuc2xhbmUubmV0PsLAlwQTAQgAQQIbAwIX
 gAIZAQULCQgHAwUVCgkICwUWAgMBAAIeBRYhBOQ+WEYd/Hy/RGkVpZn6f8tZ/DuBBQJoGNGd
 BQkdEO8nAAoJEJn6f8tZ/DuBq74H/jkTR4Zi3stbw+xC7v2u3QozssK7MYPL2AsVfh7OealS
 h182fiWXpfvmmAB7WUHbhk9GC2RAOnHI/2d2jgKaMLAHsGYOT0YopTVIwRY43fCw/mK67yxc
 wmDcX+zyKfLaivNbf5A7QPLNwda98bEAMSJ8Sn652Uc6cA8t3uKGsVzbRBQOoYzjgvBCfSrE
 9ql3PDNg0l4BfAqabd2f70ZUm9VAMEPrgv/v2xI7M2XiL4g5BVmqLCOwxLM8RMCotCuoweUr
 VO43DeBCIDwLxotMJKvGWDjBzQYlU1NPUAtNcz/gN9ITUe1VUGjyvGj4u1lxBOcQQUw7l1+T
 5moZ4iZxXzvOwE0ETspYWQEIANGc4zQULOxhbqO2dyD51YhqCNRmm9oKWaqf+wmW4tpDe/VV
 cxAnNizd4LWCHfzpb5cHAtGkOPePMfzWVf6nvdF7d3eglbtf59+zG7O7llV0xSSoFiieQBsr
 GvqDInXYX/4mRRXMtyhM353/tixC9RWLs1oofyYmCPPXXY7h9R7en3B8BoVrRFcdzlIY/NFN
 hFGW/9dkEiGjgna2Rk6e15kln4ZvFBWUg23p93w/pqXcxY6+k/8TEk+C4R+M6w7o2PLGOjdZ
 +kPiUcw5H85zf/yZJwQXzisXaNduwWB6Vads9YC9dj6kPR1c4VGRqAaYL++LAEOqrlvm2Tvq
 QqZRtnEAEQEAAcLAfAQYAQgAJgIbDBYhBOQ+WEYd/Hy/RGkVpZn6f8tZ/DuBBQJoGNI2BQkd
 EODdAAoJEJn6f8tZ/DuBfw0IAKTsfD40teP/pp+bsLLMSxPXUYrrprTj7WFB5v61p6dkpSr/
 qXmMlyahdxQFaPmfVgVirB1Vk/kHiWNnnGjfUV9nB2Zg9LI0Xb9/ts3LsUiRWXzG3tkMY6XL
 vsVOxW4XFRND9l2q+WW93aZ1DZl+fqWfYgMvsusFRhmGFOKTRfKPta2Pkv+AhA24N4+PrR5p
 bU4k2MO8PAGiK8eaYKGFG1bHKuAvoDoF7WXJ3FHxuWqLnKEt4dfOLm5pAe3zq1Lt6q8azT9i
 QWGpSAK5vQUWQHBHpiDjdPeqKZ6HiAXIIKfSmb+jrvXBqoP+D6/K7rUjG2aXiRtTIAXms9sm
 VRu7cmw=
In-Reply-To: <e4twyomp2vc2trbhqa4uszjobyeccctid4m3fzuzokxc5qdvri@mkghlmmobjgn>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/0771308b-03f4-455a-9748-a2e0cd4aab03%40dunslane.net>
Precedence: bulk


On 2026-04-01 We 9:26 AM, Andres Freund wrote:
> Hi,
>
> On 2026-04-01 06:39:05 -0400, Andrew Dunstan wrote:
>> On 2026-03-31 Tu 10:05 PM, Thomas Munro wrote:
>>> On Mon, Mar 30, 2026 at 11:23 AM Tom Lane<tgl@sss.pgh.pa.us> wrote:
>>>> Thomas Munro<thomas.munro@gmail.com> writes:
>>>>> Anyway, given the defaults, GNU tar + ZFS/BTRFS users must be pretty
>>>>> unlikely to hit this in the wild, and the symptom is a confusing error
>>>>> in a maintenance tool, not corruption, so I don't think this is a big
>>>>> deal.  I might still try teaching the astreamer code to understand PAX
>>>>> 1.0 when it sees it in the next cycle though, for the benefit of
>>>>> FreeBSD users.
>>>> I agree that this isn't too critical if the effects are confined to
>>>> pg_waldump.  I believe that pg_basebackup and pg_verifybackup also use
>>>> astreamer_tar.c, but it's not clear to me if they'd ever be asked to
>>>> parse files made by tar(1) and not by our own sparseness-ignorant
>>>> tar-writing code.  If they can be, that'd be a higher-priority reason
>>>> to fill in this gap.
>>> I pushed the workaround for the test.
>>
>> It occurred to me this morning that we probably shouldn't run this test on
>> Windows, and if we do we shouldn't be using /dev/null (the Windows
>> equivalent of which is just "nul"). The simplest fix would just be to add a
>> "!$windows_os" to the if test.
> Why should we skip this test on windows?
>
> I think we have historically been way too liberal about sprinkling
> !$windows_os test disablements around. More than once there were actual bugs
> that we just swept under the rug by disabling the tests that detected them.
> Either we support windows or we don't.
>

Maybe I misunderstood, but I didn't think this was going to be an issue 
on NTFS.

In general I agree with you, though. I try to avoid skipping things on 
Windows.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com