public inbox for [email protected]
help / color / mirror / Atom feedFrom: Amit Kapila <[email protected]>
To: Masahiko Sawada <[email protected]>
Cc: [email protected] <[email protected]>
Cc: Peter Smith <[email protected]>
Cc: [email protected] <[email protected]>
Cc: Dilip Kumar <[email protected]>
Cc: [email protected] <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: Perform streaming logical transactions by background workers and parallel apply
Date: Mon, 26 Dec 2022 18:59:44 +0530
Message-ID: <CAA4eK1KSnSAv2R8MK9puHBmihphmFcbiOXo-8rpgUnnRcqmXMw@mail.gmail.com> (raw)
In-Reply-To: <CAD21AoDvT+Tv3auBBShk19EkKLj6ByQtnAzfMjh49BhyT7f4Nw@mail.gmail.com>
References: <CAA4eK1KdVRffYEYiv9=4tVWhP=Jkfh1h6FM0A1R0NEe8cLT8+g@mail.gmail.com>
<CAA4eK1JU4J3-Z=fcMcyQw-iG-DNr=u6w_g-iM_4xwg1pKupkFQ@mail.gmail.com>
<OS0PR01MB57163693482C5470C41914AF94EA9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAHut+PuYJz-26QWMSRN_HwiJNfYG4ZxEV3RGE-VyZtXXeFm+cw@mail.gmail.com>
<OS0PR01MB5716C1C1D8244BA86311193894EB9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1LrADDNXKRVYox4Q+zSFQvtiQu-7piyd_PCmVjg1gsN2w@mail.gmail.com>
<OS0PR01MB57169EF567017861EACBD10D94E99@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB571623EFD8903C93424EFADC94E99@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB57166C4E79559C9A5570838C94EC9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAD21AoDvT+Tv3auBBShk19EkKLj6ByQtnAzfMjh49BhyT7f4Nw@mail.gmail.com>
On Mon, Dec 26, 2022 at 6:33 PM Masahiko Sawada <[email protected]> wrote:
>
> ---
> + if (!pa_can_start(xid))
> + return;
> +
> + /* First time through, initialize parallel apply worker state
> hashtable. */
> + if (!ParallelApplyTxnHash)
> + {
> + HASHCTL ctl;
> +
> + MemSet(&ctl, 0, sizeof(ctl));
> + ctl.keysize = sizeof(TransactionId);
> + ctl.entrysize = sizeof(ParallelApplyWorkerEntry);
> + ctl.hcxt = ApplyContext;
> +
> + ParallelApplyTxnHash = hash_create("logical
> replication parallel apply workershash",
> +
> 16, &ctl,
> +
> HASH_ELEM |HASH_BLOBS | HASH_CONTEXT);
> + }
> +
> + /*
> + * It's necessary to reread the subscription information
> before assigning
> + * the transaction to a parallel apply worker. Otherwise, the
> leader may
> + * not be able to reread the subscription information if streaming
> + * transactions keep coming and are handled by parallel apply workers.
> + */
> + maybe_reread_subscription();
>
> pa_can_start() checks if the skiplsn is an invalid xid or not, and
> then maybe_reread_subscription() could update the skiplsn to a valid
> value. As the comments in pa_can_start() says, it won't work. I think
> we should call maybe_reread_subscription() in
> apply_handle_stream_start() before calling pa_allocate_worker().
>
But I think a similar thing can happen when we start the worker and
then before the transaction ends, we do maybe_reread_subscription(). I
think we should try to call maybe_reread_subscription() when we are
reasonably sure that we are going to enter parallel mode, otherwise,
anyway, it will be later called by the leader worker.
--
With Regards,
Amit Kapila.
view thread (625+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Perform streaming logical transactions by background workers and parallel apply
In-Reply-To: <CAA4eK1KSnSAv2R8MK9puHBmihphmFcbiOXo-8rpgUnnRcqmXMw@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox