Message-ID: <c1e5ccd0-9681-4959-8c8a-ad4853064e98@enterprisedb.com>
Date: Wed, 26 Jun 2024 13:27:17 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: long-standing data loss bug in initial sync of logical
 replication
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Andres Freund <andres@anarazel.de>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: <de52b282-1166-1180-45a2-8d8917ca74c6@enterprisedb.com>
 <20231118015443.sfxfbr4eqolxbmjc@awork3.anarazel.de>
 <20231118025445.crhaeeuvoe2g5dv6@awork3.anarazel.de>
 <49958b83-009a-9360-2969-ea42cd15fff7@enterprisedb.com>
 <20231118181257.qtebsnkrzj3icg36@awork3.anarazel.de>
 <3c5ad91c-330a-0c5a-974a-da3801ffc18b@enterprisedb.com>
 <20231118210519.yborkczu2aln6zdd@awork3.anarazel.de>
 <5cc4bbc8-b2ad-3e6e-9878-cef8ddde8ffe@enterprisedb.com>
 <20231119021830.d6t6aaxtrkpn743y@awork3.anarazel.de>
 <CAA4eK1+W_9zQtaXX4fsrHDrLOZyoFheeLo6ExQX-U6EPGTAqEA@mail.gmail.com>
 <571f7387-edfa-4733-a335-ccce7ce01574@enterprisedb.com>
 <CAA4eK1LP2nX7u7Okk5kxu3rsZRzugrQh9vmeO32yXgrq+oxAFQ@mail.gmail.com>
Content-Language: en-US
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
In-Reply-To: 
 <CAA4eK1LP2nX7u7Okk5kxu3rsZRzugrQh9vmeO32yXgrq+oxAFQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/c1e5ccd0-9681-4959-8c8a-ad4853064e98%40enterprisedb.com>
Precedence: bulk

On 6/25/24 07:04, Amit Kapila wrote:
> On Mon, Jun 24, 2024 at 8:06 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 6/24/24 12:54, Amit Kapila wrote:
>>> ...
>>>>
>>>>>> I'm not sure there are any cases where using SRE instead of AE would cause
>>>>>> problems for logical decoding, but it seems very hard to prove. I'd be very
>>>>>> surprised if just using SRE would not lead to corrupted cache contents in some
>>>>>> situations. The cases where a lower lock level is ok are ones where we just
>>>>>> don't care that the cache is coherent in that moment.
>>>>
>>>>> Are you saying it might break cases that are not corrupted now? How
>>>>> could obtaining a stronger lock have such effect?
>>>>
>>>> No, I mean that I don't know if using SRE instead of AE would have negative
>>>> consequences for logical decoding. I.e. whether, from a logical decoding POV,
>>>> it'd suffice to increase the lock level to just SRE instead of AE.
>>>>
>>>> Since I don't see how it'd be correct otherwise, it's kind of a moot question.
>>>>
>>>
>>> We lost track of this thread and the bug is still open. IIUC, the
>>> conclusion is to use SRE in OpenTableList() to fix the reported issue.
>>> Andres, Tomas, please let me know if my understanding is wrong,
>>> otherwise, let's proceed and fix this issue.
>>>
>>
>> It's in the commitfest [https://commitfest.postgresql.org/48/4766/] so I
>> don't think we 'lost track' of it, but it's true we haven't done much
>> progress recently.
>>
> 
> Okay, thanks for pointing to the CF entry. Would you like to take care
> of this? Are you seeing anything more than the simple fix to use SRE
> in OpenTableList()?
> 

I did not find a simpler fix than adding the SRE, and I think pretty
much any other fix is guaranteed to be more complex. I don't remember
all the details without relearning all the details, but IIRC the main
challenge for me was to convince myself it's a sufficient and reliable
fix (and not working simply by chance).

I won't have time to look into this anytime soon, so feel free to take
care of this and push the fix.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company