public inbox for [email protected]
help / color / mirror / Atom feedFrom: Vitaly Davydov <[email protected]>
To: Amit Kapila <[email protected]>
Cc: Alexander Korotkov <[email protected]>
Cc: pgsql-hackers <[email protected]>
Cc: Masahiko Sawada <[email protected]>
Cc: [email protected]
Subject: Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly
Date: Mon, 09 Jun 2025 19:09:58 +0300
Message-ID: <28c8bf-68470780-3-51b29480@89454035> (raw)
In-Reply-To: <CAA4eK1LvssAcp61qg957xRF0=K4xF0hfJQjQ6c9V7EGHSqraPg@mail.gmail.com>
References: <1d12d2-67235980-35-19a406a0@63439497>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<135e8-675c4600-21-304d7f40@228322535>
<1538a2-67c5c700-7-77ec5a80@179382871>
<CAPpHfdvQPndPMbPvrHqUYbi_HnTZxxaZ4cafvF+9Mx56v07W-w@mail.gmail.com>
<27589a-680a4b80-6d-6e07880@117064404>
<CAPpHfdujSUM6de2ZD6gP_3vjUonHgiqh7StPZ5AQz-=4-qhuzA@mail.gmail.com>
<CAD21AoAbP8NJN3UZXA=RMB_LMwWFDwkV6Erzf7WpAa=jqs0_6A@mail.gmail.com>
<CAPpHfdtgwhFFpb8v8x2Humdj3F_DjaeBWY_h20g-Owh5tMArCA@mail.gmail.com>
<CAD21AoDp4Zabk49G94TdWrthfv+0mhHLA6yw-AJxxRSFWPep+g@mail.gmail.com>
<3ea6a1-68150580-1-7b53be0@80017081>
<CAPpHfdvpcboCAxbj+xPmCWNRZxG=JwW3qRsz4No_OvP3j03ERg@mail.gmail.com>
<309f08-682ca380-25-139bf240@62751029>
<CAPpHfduDLCrQ=vp1KBJu5fnSzgQp9ebGRV7i2poHK9VthwaL7g@mail.gmail.com>
<CAPpHfdutKQxpm-gJgiZRb2ouKC9+HZx3fG3F00zd=xdxDidm_g@mail.gmail.com>
<CAA4eK1+eebaviZ9TCU4DHAOZzRRbWe0HgdjjQ1VVu7P_pEsiYQ@mail.gmail.com>
<CAPpHfdsk9_mrZaR_twwe6ntFUYCwu9SM1t4n6SmWNTZDZGefoQ@mail.gmail.com>
<CAA4eK1KMaPA5jir_SFu+qr3qu55OOdFWVZpuUkqTSGZ9fyPpHA@mail.gmail.com>
<2c1d-68344100-b7-3411b8c0@256938178>
<CAA4eK1LmVoZ8jm5Jt3DERN9Z=y6L6xj5O0asw0smWw31Y8KfVw@mail.gmail.com>
<CAPpHfdtp61pSt72KxxOnnV_Oj4OTAiKYCSTek0Pm87hVDod4Bg@mail.gmail.com>
<CAA4eK1Lmd-stXWrNqX9iXugjY=ahhe7zRfyq+-vN512xvF4G2w@mail.gmail.com>
<CAPpHfdsTiYQPywr4yeqVhUtA+eFdqisqcAOiTb+kiYhi9EVEvw@mail.gmail.com>
<CAPpHfdvk-nfu5xzJyPArxWctofWPR+1L9uJ+yyWFA34n6b5rGQ@mail.gmail.com>
<CAA4eK1Ladnm40UO-YC+c7eBYqjyFQSwg0dn1w_CiaMoBTsJUDA@mail.gmail.com>
<CAPpHfduX--KKX853UwaJ8Cjo5bxbh19V_+McKQ3p9aS6b8T1Wg@mail.gmail.com>
<CAA4eK1+VbBxYY-rG4B7Jq6TsYvdjt3d2nbUrs7p9-1sP5mH+sw@mail.gmail.com>
<CAPpHfdvxqjE+RZdoOODQZAmQip6NpvSH40DuvVFWuf_00ek-yQ@mail.gmail.com>
<1d97ec-6841b600-3-b7b4780@40747246>
<CAA4eK1LvssAcp61qg957xRF0=K4xF0hfJQjQ6c9V7EGHSqraPg@mail.gmail.com>
Hi Amit,
> I think we can use this approach for HEAD and probably keep the
> previous idea for backbranches. Keeping some value in shared_memory
> per slot sounds risky to me in terms of introducing new bugs.
Not sure, what kind of problems may occur. I propose to allocate in shmem an
array of last_saved_restart_lsn like below which is not a part of the public
api (see below). It will be allocated and deallocated in shmem the same way as
ReplicationSlotCtlData. I can prepare a patch, if needed.
typedef struct ReplicationSlotCtlDataExt {
XLogRecPtr last_saved_restart_lsn[1];
} ReplicationSlotCtlDataExt;
> Yeah, but with physical slots, it is possible that the slot's xmin
> value is pointing to some value, say 700 (after restart), but vacuum
> would have removed tuples from transaction IDs greater than 700 as
> explained in email [1].
I think, we have no xmin problem for physical slots. The xmin values of
physical slots are used to process HSF messages. If I correctly understood what
you mean, you are telling about the problem which is solved by hot standby
feedback messages. This message is used to disable tuples vacuuming on the
primary to avoid delete conflicts on the replica in queries (some queries may
select some tuples which were vacuumed on the primary and deletions are
replicated to the standby). If the primary receives a HSF message after slot
saving, I believe, it is allowable if autovacuum cleans tuples with xmin later
than the last saved value. If the primary restarts, the older value will be
loaded but the replica already confirmed the newer value. Concerning replica,
it is the obligation of the replica to send such HSF xmin that will survive
replica's immediate restart.
>> Taking into account these thoughts, I can't see any problems with the alternative
>> patch where oldest wal lsn is calculated only in checkpoint.
>>
>The alternative will needlessly prevent removing WAL segments in some
>cases when logical slots are in use.
IMHO, I'm not sure, it will significantly impact the wal removal. We remove WAL
segments only in checkpoint. The alternate solution gets the oldest WAL segment
at the beginning of checkpoint, then saves dirty slots to disk, and removes old
WAL segments at the end of checkpoint using the oldest WAL segment obtained at
the beginning of checkpoint. The alternate solution may not be so effective
in terms of WAL segments removal, if a logical slot is advanced during
checkpoint, but I do not think it is a significant issue. From the other hand,
the alternate solution simplifies the logic of WAL removal, backward compatible
(avoids addition new in-memory states), decreases the number of locks in
ReplicationSlotsComputeRequiredLSN - no need to recalculate oldest slots'
restart lsn every time when a slot is advanced.
With best regards,
Vitaly
view thread (73+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly
In-Reply-To: <28c8bf-68470780-3-51b29480@89454035>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox