pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Nathan Bossart <nathandbossart@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: cleanup patches for incremental backup
Date: Wed, 24 Jan 2024 12:05:15 -0600
Message-ID: <20240124180515.GA2709722@nathanxps13> (raw)
In-Reply-To: <CA+TgmoZusu5g3rM1k=UB29Sf53c1OKjm1uFw9uo-sSXBLFZJiQ@mail.gmail.com>
References: <CAEze2Who3iG9QXbV9gbWp3WvuwSh5D65Dzv9ktZEyxpRb_QvkQ@mail.gmail.com>
	<CA+TgmoZd=pYk0owL4W2f4to9qFjwq_NmGJzXg44pEcMxuYriCA@mail.gmail.com>
	<CAEze2WizrxyWg+xeVMuyVqfZc-8U1myy1V4H3zwV-7xXztVf2w@mail.gmail.com>
	<CA+TgmoadPUdt8ZJ_Zem-T2_qZ5OTXVdxH6wGtnaV-TunN+LOHQ@mail.gmail.com>
	<CAEze2Wjbf==0=v+Ck8B2vcCeqD_3_SfDV3HtpEQiP-r3e2fsSA@mail.gmail.com>
	<CA+TgmobTfzrnBesboXutZmrABdP79mO2HhXupY5YOdZ9VLWFog@mail.gmail.com>
	<CAEze2WjhdVCqEe_qqEok3NA6DwUdOGSBjAxzmYdAqiaaH1uRcg@mail.gmail.com>
	<CA+TgmoaQHtGNFczJzSO5sMobUpuikSHphtFPZpdLmjzwdmM1DQ@mail.gmail.com>
	<20240124170846.GA2643050@nathanxps13>
	<CA+TgmoZusu5g3rM1k=UB29Sf53c1OKjm1uFw9uo-sSXBLFZJiQ@mail.gmail.com>

On Wed, Jan 24, 2024 at 12:46:16PM -0500, Robert Haas wrote:
> The "examining summary" line is generated based on the output of
> pg_available_wal_summaries(). The way that works is that the server
> calls readdir(), disassembles the filename into a TLI and two LSNs,
> and returns the result. Then, a fraction of a second later, the test
> script reassembles those components into a filename and finds the file
> missing. If the logic to translate between filenames and TLIs & LSNs
> were incorrect, the test would fail consistently. So the only
> explanation that seems to fit the facts is the file disappearing out
> from under us. But that really shouldn't happen. We do have code to
> remove such files in MaybeRemoveOldWalSummaries(), but it's only
> supposed to be nuking files more than 10 days old.
> 
> So I don't really have a theory here as to what could be happening. :-(

There might be an overflow risk in the cutoff time calculation, but I doubt
that's the root cause of these failures:

	/*
	 * Files should only be removed if the last modification time precedes the
	 * cutoff time we compute here.
	 */
	cutoff_time = time(NULL) - 60 * wal_summary_keep_time;

Otherwise, I think we'll probably need to add some additional logging to
figure out what is happening...

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com





view thread (40+ messages)  latest in thread

Message-ID: <20240124180515.GA2709722@nathanxps13>
Permalink:  ../20240124180515.GA2709722@nathanxps13/
Also on:    postgresql.org/message-id/20240124180515.GA2709722@nathanxps13

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: nathandbossart@gmail.com, robertmhaas@gmail.com, boekewurm+postgres@gmail.com, pgsql-hackers@lists.postgresql.org
  Subject: Re: cleanup patches for incremental backup
  In-Reply-To: <20240124180515.GA2709722@nathanxps13>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox