pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: David Steele <david@pgmasters.net>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: remove more archiving overhead
Date: Thu, 7 Jul 2022 14:07:26 -0400
Message-ID: <58b2ae42-5378-778f-e198-3dcaf5ecc47a@pgmasters.net> (raw)
In-Reply-To: <20220707161825.GA2205004@nathanxps13>
References: <20220222011948.GA3850532@nathanxps13>
	<20220222173711.GA3852671@nathanxps13>
	<20220222195229.GA42988@nathanxps13>
	<20220224.141349.1809640551882183816.horikyota.ntt@gmail.com>
	<20220224175553.GA660690@nathanxps13>
	<20220407222327.GA1363808@nathanxps13>
	<768e6cb3-256d-9c0b-1797-62420ffca7ae@oss.nttdata.com>
	<CA+TgmoaeTe4oUT39A4nt=8LD6UJ5u0vcmGc5+Aksn-4oKRb8-w@mail.gmail.com>
	<17147b74-5465-9a7a-ae27-305d1200482c@pgmasters.net>
	<20220707161825.GA2205004@nathanxps13>

On 7/7/22 12:18, Nathan Bossart wrote:
> On Thu, Jul 07, 2022 at 10:46:23AM -0400, David Steele wrote:
> 
>> There are plenty of ways that already-archived WAL might get archived again
>> and this is just one of them.
> 
> What are some of the others?  I was aware of the case that was fixed in
> ff9f111, where we might try to re-archive a file with different contents,
> but I'm curious what other ways you've seen this happen.

On the PG side, crashes and (IIRC) immediate shutdown.

In general, any failure in the archiver itself. Storage, memory, 
network, etc. There are plenty of ways that the file might make it to 
storage but postgres never gets notified, so it will retry.

Any archiver that is not tolerant of this fact is not going to be very 
useful and this patch only makes it slightly more true.

Regards,
-David





view thread (40+ messages)  latest in thread

Message-ID: <58b2ae42-5378-778f-e198-3dcaf5ecc47a@pgmasters.net>
Permalink:  ../58b2ae42-5378-778f-e198-3dcaf5ecc47a@pgmasters.net/
Also on:    postgresql.org/message-id/58b2ae42-5378-778f-e198-3dcaf5ecc47a@pgmasters.net

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: david@pgmasters.net, nathandbossart@gmail.com, robertmhaas@gmail.com, masao.fujii@oss.nttdata.com, horikyota.ntt@gmail.com
  Subject: Re: remove more archiving overhead
  In-Reply-To: <58b2ae42-5378-778f-e198-3dcaf5ecc47a@pgmasters.net>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox