public inbox for [email protected]
help / color / mirror / Atom feedFrom: Tomas Vondra <[email protected]>
To: Amit Kapila <[email protected]>
Cc: [email protected] <[email protected]>
Cc: Peter Eisentraut <[email protected]>
Cc: Alvaro Herrera <[email protected]>
Cc: Justin Pryzby <[email protected]>
Cc: Rahila Syed <[email protected]>
Cc: Peter Smith <[email protected]>
Cc: pgsql-hackers <[email protected]>
Cc: [email protected] <[email protected]>
Subject: Re: Column Filtering in Logical Replication
Date: Tue, 29 Mar 2022 13:03:34 +0200
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAA4eK1JNrbhSsq4NMbVr+6sBOEaLOssuefLCL1HjKHX0xD5dRQ@mail.gmail.com>
References: <[email protected]>
<[email protected]>
<CAA4eK1KUpOnFam+2bBuGGX-_pZbmO_ShJV6G=GT-RyM7gQSvWg@mail.gmail.com>
<[email protected]>
<OS0PR01MB57164514E1DE1E579B0B6C46940F9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1KvFAygPbFu=cUvnqhmTLPpjzqp8p4UNSsLkiZmvFp8oQ@mail.gmail.com>
<[email protected]>
<[email protected]>
<[email protected]>
<CAA4eK1LpBFU49Ohbnk=dv_v9YP+Kqh1+Sf8i++_s-QhD1Gy4Qw@mail.gmail.com>
<[email protected]>
<[email protected]>
<CAA4eK1JzzoE61CY1qi9Vcdi742JFwG4YA3XpoMHwfKNhbFic6g@mail.gmail.com>
<CAA4eK1JcQRQw0G-U4A+vaGaBWSvggYMMDJH4eDtJ0Yf2eUYXyA@mail.gmail.com>
<[email protected]>
<CAA4eK1JNrbhSsq4NMbVr+6sBOEaLOssuefLCL1HjKHX0xD5dRQ@mail.gmail.com>
On 3/29/22 12:00, Amit Kapila wrote:
> On Sun, Mar 20, 2022 at 4:53 PM Tomas Vondra
> <[email protected]> wrote:
>>
>> On 3/20/22 07:23, Amit Kapila wrote:
>>> On Sun, Mar 20, 2022 at 8:41 AM Amit Kapila <[email protected]> wrote:
>>>>
>>>> On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra
>>>> <[email protected]> wrote:
>>>>
>>>>> So the question is why those two sync workers never complete - I guess
>>>>> there's some sort of lock wait (deadlock?) or infinite loop.
>>>>>
>>>>
>>>> It would be a bit tricky to reproduce this even if the above theory is
>>>> correct but I'll try it today or tomorrow.
>>>>
>>>
>>> I am able to reproduce it with the help of a debugger. Firstly, I have
>>> added the LOG message and some While (true) loops to debug sync and
>>> apply workers. Test setup
>>>
>>> Node-1:
>>> create table t1(c1);
>>> create table t2(c1);
>>> insert into t1 values(1);
>>> create publication pub1 for table t1;
>>> create publication pu2;
>>>
>>> Node-2:
>>> change max_sync_workers_per_subscription to 1 in potgresql.conf
>>> create table t1(c1);
>>> create table t2(c1);
>>> create subscription sub1 connection 'dbname = postgres' publication pub1;
>>>
>>> Till this point, just allow debuggers in both workers just continue.
>>>
>>> Node-1:
>>> alter publication pub1 add table t2;
>>> insert into t1 values(2);
>>>
>>> Here, we have to debug the apply worker such that when it tries to
>>> apply the insert, stop the debugger in function apply_handle_insert()
>>> after doing begin_replication_step().
>>>
>>> Node-2:
>>> alter subscription sub1 set pub1, pub2;
>>>
>>> Now, continue the debugger of apply worker, it should first start the
>>> sync worker and then exit because of parameter change. All of these
>>> debugging steps are to just ensure the point that it should first
>>> start the sync worker and then exit. After this point, table sync
>>> worker never finishes and log is filled with messages: "reached
>>> max_sync_workers_per_subscription limit" (a newly added message by me
>>> in the attached debug patch).
>>>
>>> Now, it is not completely clear to me how exactly '013_partition.pl'
>>> leads to this situation but there is a possibility based on the LOGs
>>> it shows.
>>>
>>
>> Thanks, I'll take a look later.
>>
>
> This is still failing [1][2].
>
> [1] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2022-03-28%2005%3A16%3A53
> [2] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=flaviventris&dt=2022-03-24%2013%3A13%3A0...
>
AFAICS we've concluded this is a pre-existing issue, not something
introduced by a recently committed patch, and I don't think there's any
proposal how to fix that. So I've put that on the back burner until
after the current CF.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
view thread (185+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Column Filtering in Logical Replication
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox