public inbox for [email protected]
help / color / mirror / Atom feedFrom: Amit Kapila <[email protected]>
To: [email protected] <[email protected]>
Cc: Masahiko Sawada <[email protected]>
Cc: [email protected] <[email protected]>
Cc: Peter Smith <[email protected]>
Cc: Dilip Kumar <[email protected]>
Cc: [email protected] <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: Perform streaming logical transactions by background workers and parallel apply
Date: Sat, 17 Dec 2022 17:45:50 +0530
Message-ID: <CAA4eK1LRqEXAmh2ZEG-8XK_xDLu2Zzdoj86v4iNXDy++B2SBnQ@mail.gmail.com> (raw)
In-Reply-To: <CAA4eK1KkrkAzO-=_k_6s5V1+uNUFkvxMK-33MaFw=JCw5Hqs3w@mail.gmail.com>
References: <CAA4eK1JQTDXTfvJ5d+L0ggG4+doyd0Xji=e0OJsb=qcn_jWALA@mail.gmail.com>
<OS0PR01MB5716D6D2765E54DC739F288E940A9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB5716997A7115715F9E4EE520940D9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1JEFVZcymteMYXhYghiX7Lb=MKuEsfxgD1VfaRrLCHyzg@mail.gmail.com>
<OS0PR01MB571663F65904D00895AD159994109@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1JuRPUY2Dx=vUToRksTmW0ptqMVT3K32g0368f_ZCk-zg@mail.gmail.com>
<OS0PR01MB5716B802A1733548A99761AE94129@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1LGKYUDFZ_jFPrU497wQf2HNvt5a+tCTpqSeWSG6kfpSA@mail.gmail.com>
<CAA4eK1Jwh7j86Egk1cye=x2R_yrTjzXGj7Fx12wVybBAEq91kA@mail.gmail.com>
<OS0PR01MB571690D6FF24E9D000ECBDBD94199@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB571639E4141EBF8A2C501E6994189@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1+ELh2vPre3JHyoeKV0A9_V7aXQD0QBPv86WEn7P_rK-g@mail.gmail.com>
<OS0PR01MB57166A377638E57823219E32941A9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1KpN=dZPcCj=TLwh3wk9uvPB-M1Jrjw7wt+-L62YrDHFQ@mail.gmail.com>
<OS0PR01MB571604D401DE7A0659BD6609941D9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1LJMpBYbgDVz=g4qig8C2bM10PK=DgW7o1zn8X426vJ+Q@mail.gmail.com>
<OS0PR01MB571663B4EC5D8BE749AACD98941E9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAD21AoD_3ta_29sopuf_KJCExverP5808HjwDeqaHDrbqbxYww@mail.gmail.com>
<OS0PR01MB5716D38323B07EB0A0D097F194E09@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1Kumnc+AR86AGJ666+4Ebfny+nMbs_JZSW3O__UEiGp6w@mail.gmail.com>
<OS0PR01MB57169DAA9A2A6E68EE5E05F094E19@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAD21AoA-i_Gr1Quo2JN56ORafL=phqD=5hUvbA40EHMRZjArQA@mail.gmail.com>
<OS0PR01MB5716E94C5E503CC53177A22994E69@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1KkrkAzO-=_k_6s5V1+uNUFkvxMK-33MaFw=JCw5Hqs3w@mail.gmail.com>
On Fri, Dec 16, 2022 at 4:34 PM Amit Kapila <[email protected]> wrote:
>
> On Fri, Dec 16, 2022 at 2:47 PM [email protected]
> <[email protected]> wrote:
> >
> > > ---
> > > + active_workers = list_copy(ParallelApplyWorkerPool);
> > > +
> > > + foreach(lc, active_workers)
> > > + {
> > > + int slot_no;
> > > + uint16 generation;
> > > + ParallelApplyWorkerInfo *winfo =
> > > (ParallelApplyWorkerInfo *) lfirst(lc);
> > > +
> > > + LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
> > > + napplyworkers =
> > > logicalrep_pa_worker_count(MyLogicalRepWorker->subid);
> > > + LWLockRelease(LogicalRepWorkerLock);
> > > +
> > > + if (napplyworkers <=
> > > max_parallel_apply_workers_per_subscription / 2)
> > > + return;
> > > +
> > >
> > > Calling logicalrep_pa_worker_count() with lwlock for each worker seems
> > > not efficient to me. I think we can get the number of workers once at
> > > the top of this function and return if it's already lower than the
> > > maximum pool size. Otherwise, we attempt to stop extra workers.
> >
> > How about we directly check the length of worker pool list here which
> > seems simpler and don't need to lock ?
> >
>
> I don't see any problem with that. Also, if such a check is safe then
> can't we use the same in pa_free_worker() as well? BTW, shouldn't
> pa_stop_idle_workers() try to free/stop workers unless the active
> number reaches below max_parallel_apply_workers_per_subscription?
>
BTW, can we move pa_stop_idle_workers() functionality to a later patch
(say into v61-0006*)? That way we can focus on it separately once the
main patch is committed.
--
With Regards,
Amit Kapila.
view thread (625+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Perform streaming logical transactions by background workers and parallel apply
In-Reply-To: <CAA4eK1LRqEXAmh2ZEG-8XK_xDLu2Zzdoj86v4iNXDy++B2SBnQ@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox