public inbox for [email protected]
help / color / mirror / Atom feedFrom: Takamichi Osumi (Fujitsu) <[email protected]>
To: 'Amit Kapila' <[email protected]>
To: Peter Smith <[email protected]>
Cc: vignesh C <[email protected]>
Cc: Kyotaro Horiguchi <[email protected]>
Cc: Hayato Kuroda (Fujitsu) <[email protected]>
Cc: [email protected] <[email protected]>
Cc: [email protected] <[email protected]>
Cc: [email protected] <[email protected]>
Cc: [email protected] <[email protected]>
Cc: [email protected] <[email protected]>
Cc: [email protected] <[email protected]>
Cc: [email protected] <[email protected]>
Subject: RE: Time delayed LR (WAS Re: logical replication restrictions)
Date: Tue, 24 Jan 2023 14:03:09 +0000
Message-ID: <TYCPR01MB837367EC597BE2A0C6FBA82DEDC99@TYCPR01MB8373.jpnprd01.prod.outlook.com> (raw)
In-Reply-To: <CAA4eK1JHYypTa8PjMzGc9e5zVw8+2jfzKPrSTRLOJF_3exzRFw@mail.gmail.com>
References: <TYCPR01MB837340F78F4A16F542589195EDFF9@TYCPR01MB8373.jpnprd01.prod.outlook.com>
<CAJpy0uC2Czf2iJ17yHf4gnMb9pVJyc2QXVe6_NfxiwHhdNLmVg@mail.gmail.com>
<TYAPR01MB5866D0527B1B8D589F1C2551F5FC9@TYAPR01MB5866.jpnprd01.prod.outlook.com>
<[email protected]>
<TYCPR01MB83739C6133B50DDA8BAD1601EDFD9@TYCPR01MB8373.jpnprd01.prod.outlook.com>
<CALDaNm2d=+WBGye015DoACY_PzrRKdmwjeZwDa+Y9LXFqnT0Ow@mail.gmail.com>
<TYCPR01MB8373F3E780D5BB60E441ECF3EDC69@TYCPR01MB8373.jpnprd01.prod.outlook.com>
<CAHut+PsydEoqQaDv861qS_366hBovYAE8RRoDu3=+sBM=yo3Dg@mail.gmail.com>
<CAHut+PtzoVdm7wj69fKDrh-KLvUHd4scdTunBsQLo97sZNFLGg@mail.gmail.com>
<TYCPR01MB8373F5162C7A0E6224670CF0EDC49@TYCPR01MB8373.jpnprd01.prod.outlook.com>
<CAHut+Puwrb_DEV_7ANSKrQr23-dAmL9vrNV8ww7XsAP+kej0uw@mail.gmail.com>
<TYCPR01MB8373BED9E390C4839AF56685EDC59@TYCPR01MB8373.jpnprd01.prod.outlook.com>
<TYCPR01MB8373A23EFC25B22CB132383AEDCB9@TYCPR01MB8373.jpnprd01.prod.outlook.com>
<CAHut+Pt15f8daHYpYXDk4-e8P85hXGLgJ2LiS=X9MwAiUc63UA@mail.gmail.com>
<CAA4eK1JHYypTa8PjMzGc9e5zVw8+2jfzKPrSTRLOJF_3exzRFw@mail.gmail.com>
On Monday, January 23, 2023 7:45 PM Amit Kapila <[email protected]> wrote:
> On Mon, Jan 23, 2023 at 1:36 PM Peter Smith <[email protected]>
> wrote:
> >
> > Here are my review comments for v19-0001.
> >
> ...
> >
> > 5. parse_subscription_options
> >
> > + /*
> > + * The combination of parallel streaming mode and min_apply_delay is
> > + not
> > + * allowed. The subscriber in the parallel streaming mode applies
> > + each
> > + * stream on arrival without the time of commit/prepare. So, the
> > + * subscriber needs to depend on the arrival time of the stream in
> > + this
> > + * case, if we apply the time-delayed feature for such transactions.
> > + Then
> > + * there is a possibility where some unnecessary delay will be added
> > + on
> > + * the subscriber by network communication break between nodes or
> > + other
> > + * heavy work load on the publisher. On the other hand, applying the
> > + delay
> > + * at the end of transaction with parallel apply also can cause
> > + issues of
> > + * used resource bloat and locks kept in open for a long time. Thus,
> > + those
> > + * features can't work together.
> > + */
> >
> > IMO some re-wording might be warranted here. I am not sure quite how
> > to do it. Perhaps like below?
> >
> > SUGGESTION
> >
> > The combination of parallel streaming mode and min_apply_delay is not
> allowed.
> >
> > Here are some reasons why these features are incompatible:
> > a. In the parallel streaming mode the subscriber applies each stream
> > on arrival without knowledge of the commit/prepare time. This means we
> > cannot calculate the underlying network/decoding lag between publisher
> > and subscriber, and so always waiting for the full 'min_apply_delay'
> > period might include unnecessary delay.
> > b. If we apply the delay at the end of the transaction of the parallel
> > apply then that would cause issues related to resource bloat and locks
> > being held for a long time.
> >
> > ~~~
> >
>
> How about something like:
> The combination of parallel streaming mode and min_apply_delay is not
> allowed. This is because we start applying the transaction stream as soon as
> the first change arrives without knowing the transaction's prepare/commit time.
> This means we cannot calculate the underlying network/decoding lag between
> publisher and subscriber, and so always waiting for the full 'min_apply_delay'
> period might include unnecessary delay.
>
> The other possibility is to apply the delay at the end of the parallel apply
> transaction but that would cause issues related to resource bloat and locks
> being held for a long time.
Thank you for providing a good description ! Adopted.
The latest patch can be seen in [1].
[1] - https://www.postgresql.org/message-id/TYCPR01MB8373DC1881F382B4703F26E0EDC99%40TYCPR01MB8373.jpnprd0...
Best Regards,
Takamichi Osumi
view thread (40+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: RE: Time delayed LR (WAS Re: logical replication restrictions)
In-Reply-To: <TYCPR01MB837367EC597BE2A0C6FBA82DEDC99@TYCPR01MB8373.jpnprd01.prod.outlook.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox