public inbox for [email protected]
help / color / mirror / Atom feedFrom: [email protected] <[email protected]>
To: [email protected] <[email protected]>
To: Peter Smith <[email protected]>
To: [email protected] <[email protected]>
Cc: Amit Kapila <[email protected]>
Cc: Dilip Kumar <[email protected]>
Cc: Masahiko Sawada <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: RE: Perform streaming logical transactions by background workers and parallel apply
Date: Fri, 28 Oct 2022 09:34:16 +0000
Message-ID: <OSZPR01MB631042582805A8E8615BC413FD329@OSZPR01MB6310.jpnprd01.prod.outlook.com> (raw)
In-Reply-To: <OS3PR01MB6275F595A86FE39A0C64CF749E319@OS3PR01MB6275.jpnprd01.prod.outlook.com>
References: <OS0PR01MB5716730C06159452335D870D947F9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1Lu-6oXMk7ZaGYLwm3CRLBuzueGbasyHnNpJxu6Mq3mmg@mail.gmail.com>
<OS3PR01MB6275F145878B4A44586C46CE9E499@OS3PR01MB6275.jpnprd01.prod.outlook.com>
<CAA4eK1LMVdS6uM7Tw7ANL0BetAd76TKkmAXNNQa0haTe2tax6g@mail.gmail.com>
<OS3PR01MB6275EFC4B707650DAB9392859E4D9@OS3PR01MB6275.jpnprd01.prod.outlook.com>
<CAHut+PuVjRgGr4saN7qwq0oB8DANHVR7UfDiciB1Q3cYN54F6A@mail.gmail.com>
<OS3PR01MB627567CF4B96A23DAFAF77269E4E9@OS3PR01MB6275.jpnprd01.prod.outlook.com>
<CAA4eK1KEf6w1azJytYnLhd2Lo-=rPkRh1Jv2ePcvs=SKqNQnAQ@mail.gmail.com>
<CAA4eK1KjGNA8T8O77rRhkv6bRT6OsdQaEy--2hNrJFCc80bN0A@mail.gmail.com>
<OS0PR01MB571620741B90C769D2727D7F94559@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAHut+PvFENKb5fcMko5HHtNEAaZyNwGhu3PASrcBt+HFoFL=Fw@mail.gmail.com>
<OS0PR01MB5716B400CD81565E868616DB945F9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS3PR01MB6275328AD1B56164C66A552D9E229@OS3PR01MB6275.jpnprd01.prod.outlook.com>
<CAA4eK1Jp4xEC_hJvanrcroo-DGX8CMjZw_D4SqZOe7Az+xrPQA@mail.gmail.com>
<OS0PR01MB57160760B34E1655718F4D1994249@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAHut+PsY0aevdVqeCUJOrRQMrwpg5Wz3-Mo+bU=mCxW2+9EBTg@mail.gmail.com>
<OS0PR01MB57160B0C0FDDCED638639696942B9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAHut+Ps0HXawMD=zQ5YUncc9kjGy+md_39Y4Fdf=sKjt-LE92g@mail.gmail.com>
<CAA4eK1+C=rDCVLS0axEx=kVeA6FTAZce+4nL_y=qZ2P+NAVjow@mail.gmail.com>
<OS0PR01MB57163F73F6FA5741F3FF2176942D9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAHut+PtU3swENfCKfc3HY=8F6VasQa=Py7joLvY9oc5k56CVGQ@mail.gmail.com>
<OS3PR01MB6275F595A86FE39A0C64CF749E319@OS3PR01MB6275.jpnprd01.prod.outlook.com>
On Tue, Oct 25, 2022 2:56 PM Wang, Wei/王 威 <[email protected]> wrote:
>
> On Tues, Oct 25, 2022 at 14:28 PM Peter Smith <[email protected]>
> wrote:
> > FYI - After a recent push, the v40-0001 patch no longer applies on the
> > latest HEAD.
> >
> > [postgres@CentOS7-x64 oss_postgres_misc]$ git apply
> > ../patches_misc/v40-0001-Perform-streaming-logical-transactions-by-
> > parall.patch
> > error: patch failed: src/backend/replication/logical/launcher.c:54
> > error: src/backend/replication/logical/launcher.c: patch does not apply
>
> Thanks for your reminder.
>
> I just rebased the patch set for review.
> The new patch set will be shared later when the comments in this thread are
> addressed.
>
I tried to write a draft patch to force streaming every change instead of
waiting until logical_decoding_work_mem is exceeded, which could help to test
streaming parallel. Attach the patch. This is based on v41-0001 patch.
With this patch, I saw a problem that the subscription option "origin" doesn't
work when using streaming parallel. That's because when the parallel apply
worker writing the WAL for the changes, replorigin_session_origin is
InvalidRepOriginId. In current patch, origin can be active only in one process
at-a-time.
To fix it, maybe we need to remove this restriction, like what we did in the old
version of patch.
Regards
Shi yu
Attachments:
[application/octet-stream] 0001-Allow-streaming-every-change-instead-of-waiting-till_patch (6.1K, ../OSZPR01MB631042582805A8E8615BC413FD329@OSZPR01MB6310.jpnprd01.prod.outlook.com/2-0001-Allow-streaming-every-change-instead-of-waiting-till_patch)
download
view thread (625+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: RE: Perform streaming logical transactions by background workers and parallel apply
In-Reply-To: <OSZPR01MB631042582805A8E8615BC413FD329@OSZPR01MB6310.jpnprd01.prod.outlook.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox