Message-ID: <d81b7c70-1453-cc92-e945-34900756fcb0@enterprisedb.com>
Date: Tue, 1 Aug 2023 17:16:02 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: logical decoding and replication of sequences, take 2
Content-Language: en-US
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Masahiko Sawada <sawada.mshk@gmail.com>,
 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
References: <c5e671bc-3400-c2f4-14d3-1ae06c462dad@enterprisedb.com>
 <CAExHW5s6kMKYHo1eQppvzZQyitvZcnAyKNU6xvbKwbPuTOXJkQ@mail.gmail.com>
 <CAExHW5vHRgjWzi6zZbgCs97eW9U7xMtzXEQK+aepuzoGDsDNtg@mail.gmail.com>
 <CAA4eK1+KFeHajszmqHKZuVHTM4cMgMuQ0xEnGiTaKJM-XGU-aA@mail.gmail.com>
 <5230cd14-7ebe-4a87-c345-782daf094847@enterprisedb.com>
 <CAA4eK1+MuR5Hj7TOpuUoSnmE02re1KUUFTzFrah6-hejQ6z4aQ@mail.gmail.com>
 <21c87ea8-86c9-80d6-bc78-9b95033ca00b@enterprisedb.com>
 <CAA4eK1KAP0BmfZG4Pcvt6=vcdWT7p5J0-MtJySPhnfj4aDYE-g@mail.gmail.com>
 <CAA4eK1L+2bie=Yj5VSZqTRnoRCsvH=5sGh4bD76fnqSkLTkNLQ@mail.gmail.com>
 <b47bc083-1d3d-05b7-d883-2f13bab58cc9@enterprisedb.com>
 <CAExHW5uCUSNTn421eH5d2pVDh-DM=SJ3sERsKu-aQrP6ZPh08g@mail.gmail.com>
 <db9c7db0-99f6-a8b1-a23f-2559f3a9a4eb@enterprisedb.com>
 <CAA4eK1+ZrdEy1GW3cDTYG4tyNFihGbM8SyYFKn=JFaKgxxXrRQ@mail.gmail.com>
 <94bbf042-7d7a-9ec5-64a9-9ffc6ddc0db3@enterprisedb.com>
 <CAA4eK1+KTL+hug2-Mcmcr_nsshwaLRjRa6arW79u5R8Jx49=uQ@mail.gmail.com>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
In-Reply-To: <CAA4eK1+KTL+hug2-Mcmcr_nsshwaLRjRa6arW79u5R8Jx49=uQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/d81b7c70-1453-cc92-e945-34900756fcb0%40enterprisedb.com>
Precedence: bulk


On 8/1/23 04:59, Amit Kapila wrote:
> On Mon, Jul 31, 2023 at 5:04 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 7/31/23 11:25, Amit Kapila wrote:
>>> ...
>>>
>>> Yeah, I also think this needs a review. This is a sort of new concept
>>> where we don't use the LSN of the slot (for cases where copy returned
>>> a larger value of LSN) or a full_snapshot created corresponding to the
>>> sync slot by Walsender. For the case of the table, we build a full
>>> snapshot because we use that for copying the table but why do we need
>>> to build that for copying the sequence especially when we directly
>>> copy it from the sequence relation without caring for any snapshot?
>>>
>>
>> We need the slot to decode/apply changes during catchup. The main
>> subscription may get ahead, and we need to ensure the WAL is not
>> discarded or something like that. This applies even if the initial sync
>> step does not use the slot/snapshot directly.
>>
> 
> AFAIK, none of these needs a full_snapshot (see usage of
> SnapBuild->building_full_snapshot). The full_snapshot tracks both
> catalog and non-catalog xacts in the snapshot where we require to
> track non-catalog ones because we want to copy the table using that
> snapshot. It is relatively expensive to build a full snapshot and we
> don't do that unless it is required. For the current usage of this
> patch, I think using CRS_NOEXPORT_SNAPSHOT would be sufficient.
> 

Yeah, you may be right we don't need a full snapshot, because we don't
need to export it. We however still need a snapshot, and it wasn't clear
to me whether you suggest we don't need the slot / snapshot at all.

Anyway, I think this is "just" a matter of efficiency, not correctness.
IMHO there are bigger questions regarding the "going back" behavior
after apply restart.

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company