public inbox for [email protected]
help / color / mirror / Atom feedFrom: Amit Kapila <[email protected]>
To: [email protected] <[email protected]>
Cc: Masahiko Sawada <[email protected]>
Cc: vignesh C <[email protected]>
Cc: Greg Nancarrow <[email protected]>
Cc: [email protected] <[email protected]>
Cc: [email protected] <[email protected]>
Cc: Alexey Lesovsky <[email protected]>
Cc: Peter Eisentraut <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Subject: Re: Skipping logical replication transactions on subscriber side
Date: Thu, 2 Dec 2021 12:18:17 +0530
Message-ID: <CAA4eK1KD_C_0LSxaYB0UbG59VOgjf4mXBeSYbVWCLXAnnuqnPw@mail.gmail.com> (raw)
In-Reply-To: <OS0PR01MB571603F90202D6709B5DF5B594689@OS0PR01MB5716.jpnprd01.prod.outlook.com>
References: <CAA4eK1KzZp5YC34exW3Yb4Vk9v5iMW0YuJCQsWeTOeF7ejLdXw@mail.gmail.com>
<CAD21AoDwoJkf9_koeiLqCepTZxJgaF1_MB=Pav4qq1DNoPSciw@mail.gmail.com>
<OS0PR01MB57169DBC045C7C5609FC7EED94629@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAD21AoDNe_O+CPucd_jQPu3gGGaCLNP+J_kSPNecTdAM8HFPww@mail.gmail.com>
<CAA4eK1KZFOUQ91M3bU+8X47Ei_C-par3EdEfOBEv3t3rwnqj5g@mail.gmail.com>
<CAD21AoCNTko0Qf-F1qVzH8Q9FZk9h_dERNs9BztehrR0SJLr3A@mail.gmail.com>
<CAA4eK1JfJBJZyTEmG-z=Uu32aMTx8d+aLLorzDXt6s=ERa_ykA@mail.gmail.com>
<CALDaNm2_WY2t2GnzW1-QL823+xk5cuC0zc4Eep8fjYeLB0jfRA@mail.gmail.com>
<CAA4eK1JheT_MTmQiGfWZJeHSEJGzd-mgPPwBwA=YWjhcTfykPA@mail.gmail.com>
<CAD21AoBu2f2R6Bc9qWpky8v_h822zdkm0419j60Ph_T9ciaVsw@mail.gmail.com>
<CAD21AoAnexmfLz5dA4mPXT10b=wt0DEsp-CcAbLCZbhuh8zZoQ@mail.gmail.com>
<OS0PR01MB5716D985D4F2430E817CE30B94689@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1K_yFm_vLyNi-unqJNYk1edNkuOrqjBRWVTF5bFaTSsQg@mail.gmail.com>
<CAD21AoAUogU_4+yPrwJeaZhOUV06USE+TFJHt5OD48O43NFvnQ@mail.gmail.com>
<CAA4eK1JOsqPHZTOeifp+Q8pWgrn1qzks3ojZnSHp-1QZFcjyQQ@mail.gmail.com>
<CAD21AoAqDVCVzSpv6DOMLqLeO+m4Jh=dKv9a8Cc4qthdpYzqFw@mail.gmail.com>
<OS0PR01MB571603F90202D6709B5DF5B594689@OS0PR01MB5716.jpnprd01.prod.outlook.com>
On Wed, Dec 1, 2021 at 11:57 AM [email protected]
<[email protected]> wrote:
>
> On Wednesday, December 1, 2021 1:23 PM Masahiko Sawada <[email protected]> wrote:
> > Okay, I've attached an updated patch. Please review it.
> >
>
> I agreed that checking the result only once makes the test more stable.
> The patch looks good to me.
>
Pushed.
Now, coming back to the skip_xid patch. To summarize the discussion in
that regard so far, we have discussed various alternatives for the
syntax like:
a. ALTER SUBSCRIPTION ... [SET|RESET] SKIP TRANSACTION xxx;
b. Alter Subscription <sub_name> SET ( subscription_parameter [=value]
[, ... ] );
c. Alter Subscription <sub_name> On Error ( subscription_parameter
[=value] [, ... ] );
d. Alter Subscription <sub_name> SKIP ( subscription_parameter
[=value] [, ... ] );
where subscription_parameter can be one of:
xid = <xid_val>
lsn = <lsn_val>
...
We didn't prefer (a) as it can lead to more keywords as we add more
options; (b) as we want these new skip options to behave and be set
differently than existing subscription properties because of the
difference in their behavior; (c) as that sounds more like an action
to be performed on a future condition (error/conflict) whereas here we
already knew that an error has happened;
As per discussion till now, option (d) seems preferable. In this, we
need to see how and what to allow as options. The simplest way for the
first version is to just allow one xid to be specified at a time which
would mean that specifying multiple xids should error out. We can also
additionally allow specifying operations like 'insert', 'update',
etc., and then relation list (list of oids). What that would mean is
that for a transaction we can allow which particular operations and
relations we want to skip.
I am not sure what exactly we can provide to users to allow skipping
initial table sync as we can't specify XID there. One option that
comes to mind is to allow specifying a combination of copy_data and
relid to skip table sync for a particular relation. We might think of
not doing anything for table sync workers but not sure if that is a
good option.
Thoughts?
--
With Regards,
Amit Kapila.
view thread (400+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Skipping logical replication transactions on subscriber side
In-Reply-To: <CAA4eK1KD_C_0LSxaYB0UbG59VOgjf4mXBeSYbVWCLXAnnuqnPw@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox