public inbox for [email protected]
help / color / mirror / Atom feedFrom: Masahiko Sawada <[email protected]>
To: Amit Kapila <[email protected]>
Cc: Zhijie Hou (Fujitsu) <[email protected]>
Cc: Alexander Lakhin <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: Perform streaming logical transactions by background workers and parallel apply
Date: Tue, 9 May 2023 11:19:47 +0900
Message-ID: <CAD21AoDytm9ziQkGty81ugsHZmzNJ_DzYVNzFPVi-pSnP97k_w@mail.gmail.com> (raw)
In-Reply-To: <CAA4eK1LkPBTgcBxKrKahFu47oSK6rDq-AAs0oy4QXH6A8UzEig@mail.gmail.com>
References: <CAA4eK1LGmZsevrqJra0V4O8oBU_eKyzm2VMpSAYQaDgC6n4fkA@mail.gmail.com>
<CAFiTN-uhnJkrUsQByBHMK268T-GEx_D8DQ6b2T+aW6RiU75pbQ@mail.gmail.com>
<OS0PR01MB571621ED532C2D7C3E01625894FB9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAFiTN-t5+_zTf86EzgCwObs1ED-P_5Ab1KMw340d3oPY_+Cwpw@mail.gmail.com>
<OS3PR01MB5718AE486227CE3ACB844C5E94F89@OS3PR01MB5718.jpnprd01.prod.outlook.com>
<CAFiTN-sgQ-K9BnMWKLR7Hm36QNjS7ZOBF6hsXJmQz5ChC0Desw@mail.gmail.com>
<CAA4eK1+P9WOFBupaTZCzYtranSH=8TSUYV=kk5xQJ-VNKnvgmQ@mail.gmail.com>
<OS0PR01MB571685A645E509267AC5F30894F99@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB57168A5A1C31BC316CB63D3C94F99@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1KK7Xty7SaCReY_b5moPURowMGOsfvhOb6xQ1EAaJAh6A@mail.gmail.com>
<CAD21AoDDbM8_HJt-nMCvcjTK8K9hPzXWqJj7pyaUvR4mm_NrSg@mail.gmail.com>
<CAA4eK1K4zjeCmZUX_L09-VhjMxfqcCa+OMmpHre7YWTY2Tgxsw@mail.gmail.com>
<CAD21AoDD_P+6SCfQ2Nc79k=WwgDMyzMXzMVfhSEiGPyvYJ4H_Q@mail.gmail.com>
<[email protected]>
<OS0PR01MB57164DF9FC5366024A1952D594659@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1+Qg79oR0LknxC05DvWfh_eiyh7eO3sDOiuVsu_dCUDHA@mail.gmail.com>
<CAD21AoDo+yUwNq6nTrvE2h9bB2vZfcag=jxWc7QxuWCmkDAqcA@mail.gmail.com>
<OS0PR01MB5716E3C106DF49B3056EF72C946F9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1+JtTXVUxKAvicuvHMj3xXj=5v=Wxfka3Jt0PYe13kNQw@mail.gmail.com>
<CAA4eK1+GGTX+qhBUEmjScVLFvAi2UsTBcaM_HUY6g5Pv5CgCew@mail.gmail.com>
<OS0PR01MB57161176B8DB0A662D6F053D94729@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1LkPBTgcBxKrKahFu47oSK6rDq-AAs0oy4QXH6A8UzEig@mail.gmail.com>
On Mon, May 8, 2023 at 8:09 PM Amit Kapila <[email protected]> wrote:
>
> On Fri, May 5, 2023 at 9:14 AM Zhijie Hou (Fujitsu)
> <[email protected]> wrote:
> >
> > On Wednesday, May 3, 2023 3:17 PM Amit Kapila <[email protected]> wrote:
> > >
> >
> > Attach another patch to fix the problem that pa_shutdown will access invalid
> > MyLogicalRepWorker. I personally want to avoid introducing new static variable,
> > so I only reorder the callback registration in this version.
> >
> > When testing this, I notice a rare case that the leader is possible to receive
> > the worker termination message after the leader stops the parallel worker. This
> > is unnecessary and have a risk that the leader would try to access the detached
> > memory queue. This is more likely to happen and sometimes cause the failure in
> > regression tests after the registration reorder patch because the dsm is
> > detached earlier after applying the patch.
> >
>
> I think it is only possible for the leader apply can worker to try to
> receive the error message from an error queue after your 0002 patch.
> Because another place already detached from the queue before stopping
> the parallel apply workers. So, I combined both the patches and
> changed a few comments and a commit message. Let me know what you
> think of the attached.
I have one comment on the detaching error queue part:
+ /*
+ * Detach from the error_mq_handle for the parallel apply worker before
+ * stopping it. This prevents the leader apply worker from trying to
+ * receive the message from the error queue that might already
be detached
+ * by the parallel apply worker.
+ */
+ shm_mq_detach(winfo->error_mq_handle);
+ winfo->error_mq_handle = NULL;
In pa_detach_all_error_mq(), we try to detach error queues of all
workers in the pool. I think we should check if the queue is already
detached (i.e. is NULL) there. Otherwise, we will end up a SEGV if an
error happens after detaching the error queue and before removing the
worker from the pool.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
view thread (105+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Perform streaming logical transactions by background workers and parallel apply
In-Reply-To: <CAD21AoDytm9ziQkGty81ugsHZmzNJ_DzYVNzFPVi-pSnP97k_w@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox