From: Bossart, Nathan <bossartn@amazon.com>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: robertmhaas@gmail.com <robertmhaas@gmail.com>
Cc: alvherre@alvh.no-ip.org <alvherre@alvh.no-ip.org>
Cc: pgsql-hackers@lists.postgresql.org <pgsql-hackers@lists.postgresql.org>
Cc: mengjuan.cmj@alibaba-inc.com <mengjuan.cmj@alibaba-inc.com>
Cc: Jakub.Wartak@tomtom.com <Jakub.Wartak@tomtom.com>
Subject: Re: prevent immature WAL streaming
Date: Thu, 26 Aug 2021 03:24:48 +0000
Message-ID: <D1901DF4-B986-402A-A128-2D1DD2637016@amazon.com> (raw)
In-Reply-To: <20210826.103243.143479712849306115.horikyota.ntt@gmail.com>
References: <202108232252.dh7uxf6oxwcy@alvherre.pgsql>
<20210824.120357.1673176579644397801.horikyota.ntt@gmail.com>
<CA+TgmobaJQoJPwRpr63STzKhKTH1FYgDAhAKC_TeaMi-AbnySw@mail.gmail.com>
<20210826.103243.143479712849306115.horikyota.ntt@gmail.com>
On 8/25/21, 5:40 PM, "Kyotaro Horiguchi" <horikyota.ntt@gmail.com> wrote:
> At Wed, 25 Aug 2021 18:18:59 +0000, "Bossart, Nathan" <bossartn@amazon.com> wrote in
>> Let's say we have the following situation (F = flush, E = earliest
>> registered boundary, and L = latest registered boundary), and let's
>> assume that each segment has a cross-segment record that ends in the
>> next segment.
>>
>> F E L
>> |-----|-----|-----|-----|-----|-----|-----|-----|
>> 1 2 3 4 5 6 7 8
>>
>> Then, we write out WAL to disk and create .ready files as needed. If
>> we didn't flush beyond the latest registered boundary, the latest
>> registered boundary now becomes the earliest boundary.
>>
>> F E
>> |-----|-----|-----|-----|-----|-----|-----|-----|
>> 1 2 3 4 5 6 7 8
>>
>> At this point, the earliest segment boundary past the flush point is
>> before the "earliest" boundary we are tracking.
>
> We know we have some cross-segment records in the regin [E L] so we
> cannot add a .ready file if flush is in the region. I haven't looked
> the latest patch (or I may misunderstand the discussion here) but I
> think we shouldn't move E before F exceeds previous (or in the first
> picture above) L. Things are done that way in my ancient proposal in
> [1].
The strategy in place ensures that we track a boundary that doesn't
change until the flush position passes it as well as the latest
registered boundary. I think it is important that any segment
boundary tracking mechanism does at least those two things. I don't
see how we could do that if we didn't update E until F passed both E
and L.
Nathan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: bossartn@amazon.com, horikyota.ntt@gmail.com, robertmhaas@gmail.com, alvherre@alvh.no-ip.org, pgsql-hackers@lists.postgresql.org, mengjuan.cmj@alibaba-inc.com, Jakub.Wartak@tomtom.com
Subject: Re: prevent immature WAL streaming
In-Reply-To: <D1901DF4-B986-402A-A128-2D1DD2637016@amazon.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox