pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Nathan Bossart <nathandbossart@gmail.com>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: Cary Huang <cary.huang@highgo.ca>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Cc: SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>
Subject: Re: Switching XLog source from archive to streaming when primary available
Date: Wed, 11 Jan 2023 16:51:14 -0800
Message-ID: <20230112005114.GB2032194@nathanxps13> (raw)
In-Reply-To: <CALj2ACUq_JebY-TSK=Z6xxj8u5oGDfO+5WE3rYYeaZRL_PbcaA@mail.gmail.com>
References: <CALj2ACUpprK2nMBAr4VMoOGL4PrSj7V6upJQFhPRirUM4a0fQg@mail.gmail.com>
	<20220916.153629.1624554489607517175.horikyota.ntt@gmail.com>
	<CALj2ACUkABNTs5kiXSM-2qW_NBpP6gC-T76=Mwy-GqQd7UE3uw@mail.gmail.com>
	<CALj2ACUg_MpcOwHZQm-9M3ScyLK-NLy9=yBhU9TfTfC0U3w1YQ@mail.gmail.com>
	<20221008215221.GA894639@nathanxps13>
	<CALj2ACUaELLztrsdD_YOA3YEqJt7RagBL4pffzK5rc0CWKmLxQ@mail.gmail.com>
	<20221009214725.GD900071@nathanxps13>
	<CALj2ACXoTHjwX9crOUFTdTJXGP=eVxxwZMh6zXuCo=d04r6ztQ@mail.gmail.com>
	<20221011031001.GC1322187@nathanxps13>
	<CALj2ACUq_JebY-TSK=Z6xxj8u5oGDfO+5WE3rYYeaZRL_PbcaA@mail.gmail.com>

On Tue, Oct 18, 2022 at 07:31:33AM +0530, Bharath Rupireddy wrote:
> In summary, the standby state machine in WaitForWALToBecomeAvailable()
> exhausts all the WAL in pg_wal before switching to streaming after
> failing to fetch from archive. The v8 patch proposed upthread deviates
> from this behaviour. Hence, attaching v9 patch that keeps the
> behaviour as-is, that means, the standby exhausts all the WAL in
> pg_wal before switching to streaming after fetching WAL from archive
> for at least streaming_replication_retry_interval milliseconds.

I think this is okay.  The following comment explains why archives are
preferred over existing files in pg_wal:

	 * When doing archive recovery, we always prefer an archived log file even
	 * if a file of the same name exists in XLOGDIR.  The reason is that the
	 * file in XLOGDIR could be an old, un-filled or partly-filled version
	 * that was copied and restored as part of backing up $PGDATA.

With your patch, we might replay one of these "old" files in pg_wal instead
of the complete version of the file from the archives, but I think that is
still correct.  We'll just replay whatever exists in pg_wal (which may be
un-filled or partly-filled) before attempting streaming.  If that fails,
we'll go back to trying the archives again.

Would you mind testing this scenario?

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com





view thread (67+ messages)  latest in thread

Message-ID: <20230112005114.GB2032194@nathanxps13>
Permalink:  ../20230112005114.GB2032194@nathanxps13/
Also on:    postgresql.org/message-id/20230112005114.GB2032194@nathanxps13

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: nathandbossart@gmail.com, bharath.rupireddyforpostgres@gmail.com, horikyota.ntt@gmail.com, cary.huang@highgo.ca, pgsql-hackers@lists.postgresql.org, satyanarlapuram@gmail.com
  Subject: Re: Switching XLog source from archive to streaming when primary available
  In-Reply-To: <20230112005114.GB2032194@nathanxps13>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox