public inbox for [email protected]  

help / color / mirror / Atom feed
From: Jaroslav Novikov <[email protected]>
To: Andrey Borodin <[email protected]>
Cc: [email protected]
Cc: Michael Paquier <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: Venkata Balaji N <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: Fujii Masao <[email protected]>
Cc: Borodin Vladimir <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Cc: [email protected]
Cc: Roman Khapov <[email protected]>
Cc: Kirill Reshke <[email protected]>
Cc: [email protected]
Subject: Re: Streaming replication and WAL archive interactions
Date: Tue, 3 Mar 2026 15:06:58 +0300
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<[email protected]>
	<[email protected]>
	<CAHGQGwGJzp-QS7BODiv1uc291gAKtjzzCPb_nzUTxYKJhLsUCA@mail.gmail.com>
	<[email protected]>
	<CAEyp7J9Hy8Q__FbGeR5skjk7d0dvLC+KLXB3JUuWrXXdJ5O+Wg@mail.gmail.com>
	<[email protected]>
	<CAB7nPqQE179yogtg+nKvdwt9KROxTyt-EjumKOMbuXQtea5r3w@mail.gmail.com>
	<[email protected]>
	<CAB7nPqS=V=LF-JZsv9eGF84joQ3jxugt4gkOK6kD1CiB98_vWg@mail.gmail.com>
	<[email protected]>
	<CA+TgmoZhcbyDBr8UdGu=y5kmnVcSwcnaBqqV=UnP+oCGiNQGeQ@mail.gmail.com>
	<CAB7nPqRhsOHSc3zyL0qw4-CRphb-9Nm4sXhRCA4iKHUhkdFysA@mail.gmail.com>
	<[email protected]>
	<CAB7nPqSak3K65B+Pzp_qGrd0rO9L=1T5FLUkPEXRqtgvV9zXEw@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<[email protected]>



> On 12 Feb 2026, at 09:56, Andrey Borodin <[email protected]> wrote:
> 
> Hi Heikki,
> 
> There’s a nearby thread [0] (about 10 years later) where I’m working on a problem your patch from this thread helps solve.
> 
> In datacenter large outages, 1–2% of clusters end up with gaps in their PITR timeline.
> In HA setups, when the primary is lost, some WAL can be missing from the archive even though it was streamed to the standby. Many HA tools (PGConsul, Patroni, etc.) try to re-archive from the standby, but those WAL files may already have been removed.
> 
> Your “shared” archive mode addresses this: the standby keeps WAL until it’s archived. archive_mode=always plus an archive tool can work, but it’s expensive. In WAL-G, for example, the archive command does a GET on the standby’s WAL, then decrypts and compares. Switching to HEAD would reduce cost in some clouds but still adds cost.
> 
> Another option is coordinating archiving outside Postgres, but that would mean building distributed coordination into the archive tool.
> 
> Shared archive mode tackles this in Postgres itself.
> 
> I’ve retrofitted your patch, incorporated ideas from the Greenplum work [1], and made some improvements.
> 
> The patchset has three parts:
> * Rebase + tests – Your original patch, rebased, with tests added.
> * Timeline switching – Correct handling of timeline switches in archive status updates.
> * Avoid directory scans – Skip scanning archive_status when possible, which was costly in WAL-G setups.
> 
> What do you think?
> 
> Best regards, Andrey Borodin.
> 
> <v4-0001-Add-archive_mode-shared-for-coordinated-WAL-archi.patch><v4-0003-Optimize-ProcessArchivalReport-to-avoid-directory.patch><v4-0002-Mark-ancestor-timeline-WAL-segments-as-archived.patch>

Hi Andrey,

Adding the missing references [0] and [1].

[0] https://www.postgresql.org/message-id/5550D20D.6090703%40iki.fi
[1] https://github.com/open-gpdb/gpdb/commit/4f2db1929df1b5eed28f33505955636096bb4e8b

Best, Jaroslav Novikov.






view thread (5+ messages)  latest in thread

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: Streaming replication and WAL archive interactions
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox