Date: Sun, 9 Oct 2022 14:47:25 -0700
From: Nathan Bossart <nathandbossart@gmail.com>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, cary.huang@highgo.ca,
	pgsql-hackers@lists.postgresql.org, satyanarlapuram@gmail.com
Subject: Re: Switching XLog source from archive to streaming when primary
 available
Message-ID: <20221009214725.GD900071@nathanxps13>
References: 
 <CALj2ACUBuc574n8g-QUQGha=J8C-cWJ4KwW1dOGe5HG0NH4s2w@mail.gmail.com>
 <20220915.172207.1940822794018442836.horikyota.ntt@gmail.com>
 <CALj2ACUpprK2nMBAr4VMoOGL4PrSj7V6upJQFhPRirUM4a0fQg@mail.gmail.com>
 <20220916.153629.1624554489607517175.horikyota.ntt@gmail.com>
 <CALj2ACUkABNTs5kiXSM-2qW_NBpP6gC-T76=Mwy-GqQd7UE3uw@mail.gmail.com>
 <CALj2ACUg_MpcOwHZQm-9M3ScyLK-NLy9=yBhU9TfTfC0U3w1YQ@mail.gmail.com>
 <20221008215221.GA894639@nathanxps13>
 <CALj2ACUaELLztrsdD_YOA3YEqJt7RagBL4pffzK5rc0CWKmLxQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CALj2ACUaELLztrsdD_YOA3YEqJt7RagBL4pffzK5rc0CWKmLxQ@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/20221009214725.GD900071%40nathanxps13>
Precedence: bulk

On Sun, Oct 09, 2022 at 02:39:47PM +0530, Bharath Rupireddy wrote:
> We can give it a chance to restore from pg_wal before switching to
> streaming to not change any behaviour of the state machine. But, not
> definitely by setting currentSource to XLOG_FROM_WAL, we basically
> never explicitly set currentSource to XLOG_FROM_WAL, other than when
> not in archive recovery i.e. InArchiveRecovery is false. Also, see the
> comment [1].
> 
> Instead, the simplest would be to just pass XLOG_FROM_WAL to
> XLogFileReadAnyTLI() when we're about to switch the source to stream
> mode. This doesn't change the existing behaviour.

It might be more consistent with existing behavior, but one thing I hadn't
considered is that it might make your proposed feature ineffective when
users are copying files straight into pg_wal.  IIUC as long as the files
are present in pg_wal, the source-switch logic won't kick in.

> Unrelated to this patch, the fact that the standby polls pg_wal is not
> documented or recommended, is not true, it is actually documented [2].
> Whether or not we change the docs to be something like [3], is a
> separate discussion.

I wonder if it would be better to simply remove this extra polling of
pg_wal as a prerequisite to your patch.  The existing commentary leads me
to think there might not be a strong reason for this behavior, so it could
be a nice way to simplify your patch.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com