Date: Wed, 18 Aug 2021 14:30:05 -0400
From: "alvherre@alvh.no-ip.org" <alvherre@alvh.no-ip.org>
To: "Bossart, Nathan" <bossartn@amazon.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
	"x4mmm@yandex-team.ru" <x4mmm@yandex-team.ru>,
	"a.lubennikova@postgrespro.ru" <a.lubennikova@postgrespro.ru>,
	"hlinnaka@iki.fi" <hlinnaka@iki.fi>,
	"matsumura.ryo@fujitsu.com" <matsumura.ryo@fujitsu.com>,
	"masao.fujii@gmail.com" <masao.fujii@gmail.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: archive status ".ready" files may be created too early
Message-ID: <202108181830.wgxahkv2eu3y@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <F46750FA-AD35-4FB2-9A6A-FFB3533C6AA2@amazon.com>
Archived-At: 
 <https://www.postgresql.org/message-id/202108181830.wgxahkv2eu3y%40alvherre.pgsql>
Precedence: bulk

On 2021-Aug-18, Bossart, Nathan wrote:

> I'll add it after XLogBackgroundFlush().

I was wondering which would be better: before or after.
XLogBackgroundFlush would do it anyway, so if you do it after then it's
not clear to me that it'd do anything (I mean we should not do any new
calls of NotifySegmentsReadyForArchive and just rely on the one in
XLogBackgroundFlush -> XLogWrite).

The advantage of doing NotifySegmentsReadyForArchive before
XLogBackgroundFlush is that the files would be created sooner, so the
archiver can be working in parallel while walwriter does its other
thing; then we'd reach the NotifySegmentsReadyForArchive in
XLogBackgroundFlush and it'd find nothing to do most of the time, which
is just fine.

> I think we'll also want to set the WAL writer's latch in case it is
> hibernating.

Yeah.  (That's another advantage of doing it in walwriter rather than
bgwriter: we don't publish bgwriter's latch anywhere AFAICS).

> Another approach could be to keep the NotifySegmentsReadyForArchive()
> call in XLogInsertRecord(), but only call it if the flush pointer is
> beyond the boundary we just registered.  Or we could only set the
> latch in XLogInsertRecord() if we detect that the flush pointer has
> advanced.

Hmm.

-- 
Álvaro Herrera           39°49'30"S 73°17'W  —  https://www.EnterpriseDB.com/