Message-ID: <9a2be101-42c9-40dd-9860-aa12f06bf0e0@dunslane.net>
Date: Fri, 20 Mar 2026 15:33:18 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
To: Amul Sul <sulamul@gmail.com>, Zsolt Parragi <zsolt.parragi@percona.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Chao Li <li.evan.chao@gmail.com>,
 Jakub Wartak <jakub.wartak@enterprisedb.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: 
 <CAAJ_b94bqdWN3h2J-PzzzQ2Npbwct5ZQHggn_QoYGhC2rn-=WQ@mail.gmail.com>
 <CAAJ_b979p7Z-dBZhxBC3m2VPkYTvYzypMTWkYg2q-e1S5F_f-Q@mail.gmail.com>
 <CAAJ_b96csGv+TvdxK-zp1Zo6zrAxOJ4n-qnxiWO1f0Lk0X0N_g@mail.gmail.com>
 <dd8f7a20-f692-47cf-be79-fd6b54d4a1a6@dunslane.net>
 <CAAJ_b95k4yWsVSbptTS0SF3thZBuQURvFCCAKwwz-0yyZhbnTQ@mail.gmail.com>
 <d84d43bb-6d9a-477e-93bd-1f61a93451cc@dunslane.net>
 <72973471-8946-40c7-8b2d-8f95540d90e2@dunslane.net>
 <CAAJ_b94hkpAJ-Q8FKjGgpbRTmmXOd-agKo1Eii4yOH2N++N36Q@mail.gmail.com>
 <401bf08a-c8f1-48e2-9a30-78deaa9fa7c5@dunslane.net>
 <CAAJ_b94jrauD_FAegTnODszjnF0O+ZFLtXhXBSPhyThRUqSNVg@mail.gmail.com>
 <CAAJ_b94fiDcsN6k27hV9772kNjEbtZG5ESQvXL1KQoUcRcxrGA@mail.gmail.com>
 <CAAJ_b95JZ7SunSeCgxB3+pC+38B5smD9y4uubAGmZTNo0xtHog@mail.gmail.com>
 <CAN4CZFMn_9wgEG0q-9CCXynZ85FzVFoyVU=okWESA42U542ajw@mail.gmail.com>
 <CAAJ_b95Oj6Kb6YGsV42Gqy=N7GuOX+FMmEtUbS7NC6BvARN2mQ@mail.gmail.com>
 <CAAJ_b96AG+p+n+DZPc4isTDR_rD_-dYfVJGXhG0k2CG+kdzU9g@mail.gmail.com>
From: Andrew Dunstan <andrew@dunslane.net>
Content-Language: en-US
Autocrypt: addr=andrew@dunslane.net; keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNJEFuZHJldyBEdW5zdGFuIDxhbmRyZXdAZHVuc2xhbmUubmV0PsLAlwQTAQgAQQIbAwIX
 gAIZAQULCQgHAwUVCgkICwUWAgMBAAIeBRYhBOQ+WEYd/Hy/RGkVpZn6f8tZ/DuBBQJoGNGd
 BQkdEO8nAAoJEJn6f8tZ/DuBq74H/jkTR4Zi3stbw+xC7v2u3QozssK7MYPL2AsVfh7OealS
 h182fiWXpfvmmAB7WUHbhk9GC2RAOnHI/2d2jgKaMLAHsGYOT0YopTVIwRY43fCw/mK67yxc
 wmDcX+zyKfLaivNbf5A7QPLNwda98bEAMSJ8Sn652Uc6cA8t3uKGsVzbRBQOoYzjgvBCfSrE
 9ql3PDNg0l4BfAqabd2f70ZUm9VAMEPrgv/v2xI7M2XiL4g5BVmqLCOwxLM8RMCotCuoweUr
 VO43DeBCIDwLxotMJKvGWDjBzQYlU1NPUAtNcz/gN9ITUe1VUGjyvGj4u1lxBOcQQUw7l1+T
 5moZ4iZxXzvOwE0ETspYWQEIANGc4zQULOxhbqO2dyD51YhqCNRmm9oKWaqf+wmW4tpDe/VV
 cxAnNizd4LWCHfzpb5cHAtGkOPePMfzWVf6nvdF7d3eglbtf59+zG7O7llV0xSSoFiieQBsr
 GvqDInXYX/4mRRXMtyhM353/tixC9RWLs1oofyYmCPPXXY7h9R7en3B8BoVrRFcdzlIY/NFN
 hFGW/9dkEiGjgna2Rk6e15kln4ZvFBWUg23p93w/pqXcxY6+k/8TEk+C4R+M6w7o2PLGOjdZ
 +kPiUcw5H85zf/yZJwQXzisXaNduwWB6Vads9YC9dj6kPR1c4VGRqAaYL++LAEOqrlvm2Tvq
 QqZRtnEAEQEAAcLAfAQYAQgAJgIbDBYhBOQ+WEYd/Hy/RGkVpZn6f8tZ/DuBBQJoGNI2BQkd
 EODdAAoJEJn6f8tZ/DuBfw0IAKTsfD40teP/pp+bsLLMSxPXUYrrprTj7WFB5v61p6dkpSr/
 qXmMlyahdxQFaPmfVgVirB1Vk/kHiWNnnGjfUV9nB2Zg9LI0Xb9/ts3LsUiRWXzG3tkMY6XL
 vsVOxW4XFRND9l2q+WW93aZ1DZl+fqWfYgMvsusFRhmGFOKTRfKPta2Pkv+AhA24N4+PrR5p
 bU4k2MO8PAGiK8eaYKGFG1bHKuAvoDoF7WXJ3FHxuWqLnKEt4dfOLm5pAe3zq1Lt6q8azT9i
 QWGpSAK5vQUWQHBHpiDjdPeqKZ6HiAXIIKfSmb+jrvXBqoP+D6/K7rUjG2aXiRtTIAXms9sm
 VRu7cmw=
In-Reply-To: 
 <CAAJ_b96AG+p+n+DZPc4isTDR_rD_-dYfVJGXhG0k2CG+kdzU9g@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/9a2be101-42c9-40dd-9860-aa12f06bf0e0%40dunslane.net>
Precedence: bulk


On 2026-03-20 Fr 9:26 AM, Amul Sul wrote:
> On Fri, Mar 20, 2026 at 5:01 PM Amul Sul <sulamul@gmail.com> wrote:
>> On Fri, Mar 20, 2026 at 2:18 AM Zsolt Parragi <zsolt.parragi@percona.com> wrote:
>>> Hello!
>>>
>>> Path is ignored with a positional argument, I think this is a bug?
>>>
>>> This fails:
>>>
>>> pg_waldump --path /wal/dir 000000010000000000000001
>>>
>>> And this works:
>>>
>>> pg_waldump --path /wal/dir --start 0/01000028 --end 0/010020F8
>>>
>> Good catch! I've fixed this in the attached version and updated a test
>> case to cover this scenario.
>>
>>> +{
>>> + int fname_len = strlen(fname);
>>> +
>>>
>>> Shouldn't this use size_t?
>>>
>> Okay, that can be used. I’ve done the same in the attached version.
>>
>>> + /*
>>> + * Setup temporary directory to store WAL segments and set up an exit
>>> + * callback to remove it upon completion.
>>> + */
>>> + setup_tmpwal_dir(waldir);
>>>
>>> Maybe this could be deferred to be created only on first use? If I
>>> understand correctly, in a typical scenario waldump won't use this
>>> temporary directory, yet it always creates it.
>> Yeah, that optimization can be done, but passing the waldir  -- which
>> is only used once -- to the point where the first temp file is created
>> would require quite a bit of code refactoring that doesn't seem to
>> offer much gain, IMO.
>>
> Since Andrew also leans toward creating the directory only when
> needed, I have reconsidered the approach. I think we can pass waldir
> (the archive directory) via XLogDumpPrivate, and I’ve implemented that
> in the attached version.
>

Thanks, committed with very minor tweaks.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com