public inbox for [email protected]
help / color / mirror / Atom feedFrom: Amit Kapila <[email protected]>
To: Masahiko Sawada <[email protected]>
Cc: [email protected] <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: Force streaming every change in logical decoding
Date: Wed, 7 Dec 2022 08:14:32 +0530
Message-ID: <CAA4eK1LOyrmw3j9BeNWSCvVbZi9odJ1ZFTGeOwuEKTwAPTwLBw@mail.gmail.com> (raw)
In-Reply-To: <CAD21AoCjeN4dAoAZX6G57LY12=UPZdFMemB+5PWX_yV=N=8NBw@mail.gmail.com>
References: <OSZPR01MB63104E7449DBE41932DB19F1FD1B9@OSZPR01MB6310.jpnprd01.prod.outlook.com>
<CAA4eK1Lt21s8cK7pMYB-2vUT79n5j+6o1MduRyNm5D=R_Y=mVw@mail.gmail.com>
<CAD21AoCjeN4dAoAZX6G57LY12=UPZdFMemB+5PWX_yV=N=8NBw@mail.gmail.com>
On Tue, Dec 6, 2022 at 7:18 PM Masahiko Sawada <[email protected]> wrote:
>
> On Tue, Dec 6, 2022 at 7:29 PM Amit Kapila <[email protected]> wrote:
> >
> > On Tue, Dec 6, 2022 at 11:53 AM [email protected]
> > <[email protected]> wrote:
> > >
> > > Hi hackers,
> > >
> > > In logical decoding, when logical_decoding_work_mem is exceeded, the changes are
> > > sent to output plugin in streaming mode. But there is a restriction that the
> > > minimum value of logical_decoding_work_mem is 64kB. I tried to add a GUC to
> > > allow sending every change to output plugin without waiting until
> > > logical_decoding_work_mem is exceeded.
> > >
> > > This helps to test streaming mode. For example, to test "Avoid streaming the
> > > transaction which are skipped" [1], it needs many XLOG_XACT_INVALIDATIONS
> > > messages. With the new option, it can be tested with fewer changes and in less
> > > time. Also, this new option helps to test more scenarios for "Perform streaming
> > > logical transactions by background workers" [2].
> > >
> >
> > Yeah, I think this can also help in reducing the time for various
> > tests in test_decoding/stream and
> > src/test/subscription/t/*_stream_*.pl file by reducing the number of
> > changes required to invoke streaming mode.
>
> +1
>
> > Can we think of making this
> > GUC extendible to even test more options on server-side (publisher)
> > and client-side (subscriber) cases? For example, we can have something
> > like logical_replication_mode with the following valid values: (a)
> > server_serialize: this will serialize each change to file on
> > publishers and then on commit restore and send all changes; (b)
> > server_stream: this will stream each change as currently proposed in
> > your patch. Then if we want to extend it for subscriber-side testing
> > then we can introduce new options like client_serialize for the case
> > being discussed in the email [1].
>
> Setting logical_replication_mode = 'client_serialize' implies that the
> publisher behaves as server_stream? or do you mean we can set like
> logical_replication_mode = 'server_stream, client_serialize'?
>
The latter one (logical_replication_mode = 'server_stream,
client_serialize'). The idea is to cover more options with one GUC and
each option can be used individually as well as in combination with
others.
--
With Regards,
Amit Kapila.
view thread (61+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: Force streaming every change in logical decoding
In-Reply-To: <CAA4eK1LOyrmw3j9BeNWSCvVbZi9odJ1ZFTGeOwuEKTwAPTwLBw@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox