From: Bossart, Nathan <bossartn@amazon.com>
To: alvherre@alvh.no-ip.org <alvherre@alvh.no-ip.org>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: x4mmm@yandex-team.ru <x4mmm@yandex-team.ru>
Cc: a.lubennikova@postgrespro.ru <a.lubennikova@postgrespro.ru>
Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>
Cc: matsumura.ryo@fujitsu.com <matsumura.ryo@fujitsu.com>
Cc: masao.fujii@gmail.com <masao.fujii@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: archive status ".ready" files may be created too early
Date: Wed, 18 Aug 2021 18:02:31 +0000
Message-ID: <F46750FA-AD35-4FB2-9A6A-FFB3533C6AA2@amazon.com> (raw)
In-Reply-To: <202108181745.bi4s5alwmadt@alvherre.pgsql>
References: <202108181740.zmdxsi3uci2k@alvherre.pgsql>
<202108181745.bi4s5alwmadt@alvherre.pgsql>
On 8/18/21, 10:48 AM, "alvherre@alvh.no-ip.org" <alvherre@alvh.no-ip.org> wrote:
> On 2021-Aug-18, alvherre@alvh.no-ip.org wrote:
>
>> I realize this means there's a contradiction with my previous argument,
>> in that synchronous transaction commit calls XLogWrite at some point, so
>> we *are* putting the client-connected backend in charge of creating the
>> notify files. However, that only happens on transaction commit, where
>> we already accept responsibility for the WAL flush, not on each
>> individual XLOG record insert; also, the WAL writer will take care of it
>> sometimes, for transactions that are long-enough lived.
>
> Eh. I just said WAL writer will sometimes do it, and that's true
> because it'll occur in XLogBackgroundFlush. But upthread I wimped out
> of having WAL writer call NotifySegmentsReadyForArchive() and instead
> opined to give responsibility to bgwriter. However, thinking about it
> again, maybe it does make sense to have walwriter do it too directly.
> This causes no harm to walwriter's time constraints, since *it will have
> to do it via XLogBackgroundFlush anyway*.
I'll add it after XLogBackgroundFlush(). I think we'll also want to
set the WAL writer's latch in case it is hibernating.
Another approach could be to keep the NotifySegmentsReadyForArchive()
call in XLogInsertRecord(), but only call it if the flush pointer is
beyond the boundary we just registered. Or we could only set the
latch in XLogInsertRecord() if we detect that the flush pointer has
advanced.
Nathan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: bossartn@amazon.com, alvherre@alvh.no-ip.org, horikyota.ntt@gmail.com, x4mmm@yandex-team.ru, a.lubennikova@postgrespro.ru, hlinnaka@iki.fi, matsumura.ryo@fujitsu.com, masao.fujii@gmail.com
Subject: Re: archive status ".ready" files may be created too early
In-Reply-To: <F46750FA-AD35-4FB2-9A6A-FFB3533C6AA2@amazon.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox