From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Nikhil Sontakke <nikhils@2ndquadrant.com>
To: Andres Freund <andres@anarazel.de>
Cc: Simon Riggs <simon@2ndquadrant.com>
Cc: Craig Ringer <craig@2ndquadrant.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Petr Jelinek <petr.jelinek@2ndquadrant.com>
Cc: Sokolov Yura <y.sokolov@postgrespro.ru>
Cc: Stas Kelvich <s.kelvich@postgrespro.ru>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Cc: Robert Haas <robertmhaas@gmail.com>
Subject: Re: [HACKERS] logical decoding of two-phase transactions
Date: Tue, 3 Apr 2018 15:56:50 +0200
Message-ID: <a76f3f12-ed44-b724-1c66-d13d1920fcab@2ndquadrant.com> (raw)
In-Reply-To: <CAMGcDxchx=0PeQBVLzrgYG2AQ49QSRxHj5DCp7yy0QrJR0S0nA@mail.gmail.com>
References: <CAMsr+YEb-f1oG+1wGji_sgswRsVQYtVXuTbVH=nnY6thMfiCqA@mail.gmail.com>
<CAMGcDxcD2fWqUwiPWGL_d-ie-CoGTVFAsgQvQ91H93q2Jc-mhA@mail.gmail.com>
<CAMGcDxf9HHmDOti4-DSQTtGo48R4OMki+BktPn2uTOtGr4XomQ@mail.gmail.com>
<c167340c-0751-d54b-0f32-fa83fd414863@2ndquadrant.com>
<CAMGcDxcGtRXtF1r3VqddzHba+BTm7q8wbgEsyhmOFuDckMTZFQ@mail.gmail.com>
<dd5a9cb7-bb10-cb5c-834e-08dc3ffc057f@2ndquadrant.com>
<20180329215815.36xxakgujx4luo4m@alap3.anarazel.de>
<0925358a-e11d-51bc-c3c1-959ded75f604@2ndquadrant.com>
<20180329222441.xxo2zxkjydpgjs4n@alap3.anarazel.de>
<CANP8+jLZy6Pxqp2Vxo20OuR5v_B1hEywR6Hp7DHwNn6Yi3B-Dw@mail.gmail.com>
<20180402180451.ip3uuvadhgxjvefo@alap3.anarazel.de>
<CAMGcDxchx=0PeQBVLzrgYG2AQ49QSRxHj5DCp7yy0QrJR0S0nA@mail.gmail.com>
On 04/03/2018 12:40 PM, Nikhil Sontakke wrote:
> Hi,
>
>>> It's certainly a nice future goal to have it all happen automatically,
>>> but we don't know what the plugin will do.
>>
>> No, fighting too complicated APIs is not unreasonable. And we've found
>> an alternative.
>>
>
> PFA, latest patch set.
>
> The LogicalLockTransaction/LogicalUnlockTransaction API implementation
> using decode groups now has proper cleanup handling in case there's an
> ERROR while holding the logical lock.
>
> Rest of the patches are the same as yesterday.
>
Unfortunately, this does segfault for me in `make check` almost
immediately. Try
./configure --enable-debug --enable-cassert CFLAGS="-O0 -ggdb3
-DRANDOMIZE_ALLOCATED_MEMORY" && make -s clean && make -s -j4 check
and you should get an assert failure right away. Examples of backtraces
attached, not sure what exactly is the issue.
Also, I get this compiler warning:
proc.c: In function ‘AssignDecodeGroupLeader’:
proc.c:1975:8: warning: variable ‘pid’ set but not used
[-Wunused-but-set-variable]
int pid;
^~~
All of PostgreSQL successfully made. Ready to install.
which suggests we don't really need the pid variable.
> Other than this, we would want to have pgoutput support for 2PC
> decoding to be made optional? In that case we could add an option to
> "CREATE SUBSCRIPTION". This will mean adding a new
> Anum_pg_subscription_subenable_twophase attribute to Subscription
> struct and related processing. Should we go down this route?
>
I'd say yes, we need to make it opt-in (assuming we want pgoutput to
support the 2PC decoding at all).
The trouble is that while it may improve replication of two-phase
transactions, it may also require config changes on the subscriber (to
support enough prepared transactions) and furthermore the GID is going
to be copied to the subscriber.
Which means that if the publisher/subscriber (at the instance level) are
already part of the are on the same 2PC transaction, it can't possibly
proceed because the subscriber won't be able to do PREPARE TRANSACTION.
So I think we need a subscription parameter to enable/disable this,
defaulting to 'disabled'.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:backtraces.txt (text/plain, 3.8K) download
#0 0x000074a570c468df in raise () from /lib64/libc.so.6
#1 0x000074a570c484da in abort () from /lib64/libc.so.6
#2 0x0000000000a14d33 in ExceptionalCondition (conditionName=0xc29608 "!(MyProc->backendId != (-1))", errorType=0xc2878a "FailedAssertion", fileName=0xc28783 "lock.c", lineNumber=4249) at assert.c:54
#3 0x0000000000893180 in VirtualXactLockTableCleanup () at lock.c:4249
#4 0x000000000088f113 in LockReleaseAll (lockmethodid=1, allLocks=false) at lock.c:2042
#5 0x0000000000894c7f in ProcReleaseLocks (isCommit=true) at proc.c:786
#6 0x0000000000a52950 in ResourceOwnerReleaseInternal (owner=0x121bf60, phase=RESOURCE_RELEASE_LOCKS, isCommit=true, isTopLevel=true) at resowner.c:564
#7 0x0000000000a5276d in ResourceOwnerRelease (owner=0x121bf60, phase=RESOURCE_RELEASE_LOCKS, isCommit=true, isTopLevel=true) at resowner.c:480
#8 0x00000000005401f3 in CommitTransaction () at xact.c:2103
#9 0x0000000000540d76 in CommitTransactionCommand () at xact.c:2768
#10 0x0000000000a29e12 in InitPostgres (in_dbname=0x1216748 "regression", dboid=0, username=0x1216728 "user", useroid=0, out_dbname=0x0) at postinit.c:1046
#11 0x00000000008b027f in PostgresMain (argc=1, argv=0x12169c0, dbname=0x1216748 "regression", username=0x1216728 "user") at postgres.c:3778
#12 0x000000000080cb9f in BackendRun (port=0x120d580) at postmaster.c:4409
#13 0x000000000080c2fd in BackendStartup (port=0x120d580) at postmaster.c:4081
#14 0x00000000008088af in ServerLoop () at postmaster.c:1754
#15 0x0000000000807ed8 in PostmasterMain (argc=8, argv=0x11e6420) at postmaster.c:1362
#16 0x000000000073ad8c in main (argc=8, argv=0x11e6420) at main.c:228
#0 0x000074a570c468df in raise () from /lib64/libc.so.6
#1 0x000074a570c484da in abort () from /lib64/libc.so.6
#2 0x0000000000a14d33 in ExceptionalCondition (conditionName=0xc25af8 "!(((allPgXact[proc->pgprocno].xid) != ((TransactionId) 0)))", errorType=0xc25ae8 "FailedAssertion", fileName=0xc25ab9 "procarray.c",
lineNumber=408) at assert.c:54
#3 0x000000000087f166 in ProcArrayEndTransaction (proc=0x74a569e07700, latestXid=1376) at procarray.c:408
#4 0x0000000000540186 in CommitTransaction () at xact.c:2061
#5 0x0000000000540d76 in CommitTransactionCommand () at xact.c:2768
#6 0x00000000008ae7fc in finish_xact_command () at postgres.c:2499
#7 0x00000000008ac4cf in exec_simple_query (query_string=0x11eb9e8 "CREATE TABLE target (tid integer, balance integer);") at postgres.c:1146
#8 0x00000000008b0798 in PostgresMain (argc=1, argv=0x12169a0, dbname=0x1216748 "regression", username=0x1216728 "user") at postgres.c:4149
#9 0x000000000080cb9f in BackendRun (port=0x120d580) at postmaster.c:4409
#10 0x000000000080c2fd in BackendStartup (port=0x120d580) at postmaster.c:4081
#11 0x00000000008088af in ServerLoop () at postmaster.c:1754
#12 0x0000000000807ed8 in PostmasterMain (argc=8, argv=0x11e6420) at postmaster.c:1362
#13 0x000000000073ad8c in main (argc=8, argv=0x11e6420) at main.c:228
#0 0x000074a570c468df in raise () from /lib64/libc.so.6
#1 0x000074a570c484da in abort () from /lib64/libc.so.6
#2 0x0000000000a14d33 in ExceptionalCondition (conditionName=0xc299c0 "!(SHMQueueEmpty(&(MyProc->myProcLocks[i])))", errorType=0xc29929 "FailedAssertion", fileName=0xc29922 "proc.c", lineNumber=395)
at assert.c:54
#3 0x0000000000894358 in InitProcess () at proc.c:395
#4 0x00000000008b0245 in PostgresMain (argc=1, argv=0x1216990, dbname=0x1216748 "regression", username=0x1216728 "user") at postgres.c:3765
#5 0x000000000080cb9f in BackendRun (port=0x120ceb0) at postmaster.c:4409
#6 0x000000000080c2fd in BackendStartup (port=0x120ceb0) at postmaster.c:4081
#7 0x00000000008088af in ServerLoop () at postmaster.c:1754
#8 0x0000000000807ed8 in PostmasterMain (argc=8, argv=0x11e6420) at postmaster.c:1362
#9 0x000000000073ad8c in main (argc=8, argv=0x11e6420) at main.c:228
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, nikhils@2ndquadrant.com, andres@anarazel.de, simon@2ndquadrant.com, craig@2ndquadrant.com, peter.eisentraut@2ndquadrant.com, petr.jelinek@2ndquadrant.com, y.sokolov@postgrespro.ru, s.kelvich@postgrespro.ru, 9erthalion6@gmail.com, tgl@sss.pgh.pa.us, sawada.mshk@gmail.com, robertmhaas@gmail.com
Subject: Re: [HACKERS] logical decoding of two-phase transactions
In-Reply-To: <a76f3f12-ed44-b724-1c66-d13d1920fcab@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox