public inbox for [email protected]
help / color / mirror / Atom feedFrom: Amul Sul <[email protected]>
To: Jakub Wartak <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
Date: Fri, 21 Nov 2025 17:14:26 +0530
Message-ID: <CAAJ_b956a+e8-HNEpeJ60ByFv7XJRqECPu3B0dozv0ChMRTPbQ@mail.gmail.com> (raw)
In-Reply-To: <CAKZiRmyDk5KqovS9Ez3iFHd+p-TChSt2QTtWkwJ5Ya-+4gg21g@mail.gmail.com>
References: <CAAJ_b94bqdWN3h2J-PzzzQ2Npbwct5ZQHggn_QoYGhC2rn-=WQ@mail.gmail.com>
<CAAJ_b97PQjE4kFD8Qk6UvtLrfPMixw1nxBz0OP5Z2WB2B-uMxQ@mail.gmail.com>
<CAAJ_b97JA8ehy_UDddrnGwDt9HG5NmJq8ATtmeMqo7YD-=tLyQ@mail.gmail.com>
<CA+TgmoZjhWDG_AR1i+L1yss-wbuWvxrdRwSdVUUUnVPrJV2CnQ@mail.gmail.com>
<CAAJ_b94Uh+b41LQG45bZFK+i62EVvv972LiGWWWuR64=-64rTQ@mail.gmail.com>
<CA+TgmobF5c7ZcZHdEhqwNxGDZzWG2bDtpRaDtoVELWX_VHs_1A@mail.gmail.com>
<CAAJ_b94gK1np8d1h-2c1YoCccGXr4zspTa-FC7X_bfXZNz=-DA@mail.gmail.com>
<CA+TgmoayDY5b+bP1vRRN7A3xOP-=+tK13B2C1g-Xm1j4WTrT9Q@mail.gmail.com>
<CAAJ_b97JAF+Zuoh2FBO79hVwLeaBPwsbXw-fY+313a7LfRQ-Bg@mail.gmail.com>
<CA+Tgmoardk4VuthHc23vov+AVkhq7eT0mFUs-2ctAnP1uiTaog@mail.gmail.com>
<CAAJ_b959x5VjmLJFmN78r_QohQuuj=fde11VbbAOHn5TzgEzng@mail.gmail.com>
<CAAJ_b97_N+0sipFyq80n0jX-nKcjcQEMOSTVg8DsqkHR8dW_Sw@mail.gmail.com>
<CAKZiRmyDk5KqovS9Ez3iFHd+p-TChSt2QTtWkwJ5Ya-+4gg21g@mail.gmail.com>
On Wed, Nov 19, 2025 at 1:50 PM Jakub Wartak
<[email protected]> wrote:
>
> On Mon, Nov 17, 2025 at 5:51 AM Amul Sul <[email protected]> wrote:
> >
> > On Thu, Nov 6, 2025 at 2:33 PM Amul Sul <[email protected]> wrote:
> > >
> > > On Mon, Oct 20, 2025 at 8:05 PM Robert Haas <[email protected]> wrote:
> > > >
> > > > On Thu, Oct 16, 2025 at 7:49 AM Amul Sul <[email protected]> wrote:
> > > > [....]
> > > Kindly have a look at the attached version. Thank you !
> > >
> >
> > Attached is the rebased version against the latest master head (e76defbcf09).
>
> Hi Amul, thanks for working on this. I haven't really looked at the
> source code deeply (I trust Robert eyes much more than mine on this
> one), just skimmed a little bit:
>
> 1. As stated earlier, get_tmp_walseg_path() is still vulnerable (it
> uses predictable path that could be used by attacker in $TMPDIR)
>
Yeah, I haven't done anything regarding this since I am unsure of what
should be done and what the risks involved are. I am thinking of
taking Robert's opinion on this.
> 2. On the usability front:
>
> a. If you do `pg_waldump --path pg_wal.tar -s 0/31000000` it will dump
> a lot of WAL records and then print final:
> pg_waldump: error: could not find file "000000010000000000000034" in archive
>
> However, with `pg_waldump --path pg_wal.tar -s 0/31000000
> --stats=record` (not passing '-e') it will simply bailout without
> printing stats and with error:
> pg_waldump: error: could not find file "000000010000000000000034" in archive
>
> IMHO, it could print stats if it was capable of getting at least 1 WAL record.
>
The similar behavior in the current pg_waldump when using the --path
option with a WAL directory and a starting LSN. E.g:
$ pg_waldump -s 0/04FE36E0 --path=/tmp/backup/tmp/ --stats=record
pg_waldump: first record is after 0/04FE36E0, at 0/04FE3F90, skipping
over 2224 bytes
pg_waldump: error: could not find file "000000010000000000000009": No
such file or directory
> 3. The most critical issue for me was the initial lack of error
> pass-through from pg_waldump (when used with WALs in tar) to the
> pg_verifybackup. Now it works fine, so thanks for this:
>
Thanks, that was exactly the intention -- to complete pg_verifybackup
for tar-formatted backup verification.
Regards,
Amul
view thread (83+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
In-Reply-To: <CAAJ_b956a+e8-HNEpeJ60ByFv7XJRqECPu3B0dozv0ChMRTPbQ@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox