From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Nikhil Sontakke <nikhils@2ndquadrant.com>
To: David Steele <david@pgmasters.net>
Cc: Andres Freund <andres@anarazel.de>
Cc: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
Cc: Simon Riggs <simon@2ndquadrant.com>
Cc: Craig Ringer <craig@2ndquadrant.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Petr Jelinek <petr.jelinek@2ndquadrant.com>
Cc: Sokolov Yura <y.sokolov@postgrespro.ru>
Cc: Stas Kelvich <s.kelvich@postgrespro.ru>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Cc: Robert Haas <robertmhaas@gmail.com>
Subject: Re: [HACKERS] logical decoding of two-phase transactions
Date: Mon, 16 Jul 2018 17:21:22 +0200
Message-ID: <0c7e9fb9-01f7-3284-da9e-22b6dde679d4@2ndquadrant.com> (raw)
In-Reply-To: <CAMGcDxdW0LMpjxwnC6_YT2UoPgY9onXAEAbJwYDeux2mSuekOw@mail.gmail.com>
References: <782517ca-1e99-06fe-eba9-34f826118289@2ndquadrant.com>
<CAMGcDxc-kuO9uq0zRCRwbHWBj_rePY9=raR7M9pZGWoj9EOGdg@mail.gmail.com>
<CAMGcDxc3sc-C928toHEvj2dH9t=wTP=BZwUv3rE1bnHzTr9R6Q@mail.gmail.com>
<9cac2837-7e58-1cad-3940-27ac0dd7c198@2ndquadrant.com>
<CAMGcDxfAnJtJ17hejKFDa3xXS2OWpys+p-PvGS+MZj4h73QqPA@mail.gmail.com>
<b1c6ba51-436c-b10d-895f-3805d3caf4a5@2ndquadrant.com>
<CAMGcDxeyO-vu4WEsX8ZZRkV7LB037Rd5EmekQuZ_bj0Y33KbBA@mail.gmail.com>
<CAMGcDxeW2xE51Rx8V8hwL7YMkVFBwtOwDco5=opWd-YACJMi0g@mail.gmail.com>
<CAA8=A7_c9jiNMzpHsH+wxEA91WUt_cRc-BN8U0S+MErhJnhZTw@mail.gmail.com>
<20180406162023.wcic5kw73bgswgb6@alap3.anarazel.de>
<CAMGcDxev9QiGpAkrNxvTKeGtYJTKRmo0UxJuQUyWGuOKgjEtdw@mail.gmail.com>
<9ea4c35b-f8ec-220b-2429-b860b49748f3@pgmasters.net>
<CAMGcDxfmy1nLtoukiHbavfqm+scvT9=WckaCtRYW1fq7LRFd9Q@mail.gmail.com>
<CAMGcDxdn8SQMuGWZ-hDVnkXv_eAbPzWRmFNU36a63b7zPeUhiw@mail.gmail.com>
<CAMGcDxdW0LMpjxwnC6_YT2UoPgY9onXAEAbJwYDeux2mSuekOw@mail.gmail.com>
Hi Nikhil,
I've been looking at this patch series, and I do have a bunch of
comments and questions, as usual ;-)
Overall, I think it's clear the main risk associated with this patch is
the decode group code - it touches PROC entries, so a bug may cause
trouble pretty easily. So I've focused on this part, for now.
1) LogicalLockTransaction does roughly this
...
if (MyProc->decodeGroupLeader == NULL)
{
leader = AssignDecodeGroupLeader(txn->xid);
if (leader == NULL ||
!BecomeDecodeGroupMember((PGPROC *)leader, txn->xid))
goto lock_cleanup;
}
leader = BackendXidGetProc(txn->xid);
if (!leader)
goto lock_cleanup;
leader_lwlock = LockHashPartitionLockByProc(leader);
LWLockAcquire(leader_lwlock, LW_EXCLUSIVE);
pgxact = &ProcGlobal->allPgXact[leader->pgprocno];
if(pgxact->xid != txn->xid)
{
LWLockRelease(leader_lwlock);
goto lock_cleanup;
}
...
I wonder why we need the BackendXidGetProc call after the first if
block. Can we simply grab MyProc->decodeGroupLeader at that point?
2) InitProcess now resets decodeAbortPending/decodeLocked flags, while
checking decodeGroupLeader/decodeGroupMembers using asserts. Isn't that
a bit strange? Shouldn't it do the same thing with both?
3) A comment in ProcKill says this:
* Detach from any decode group of which we are a member. If the leader
* exits before all other group members, its PGPROC will remain allocated
* until the last group process exits; that process must return the
* leader's PGPROC to the appropriate list.
So I'm wondering what happens if the leader dies before other group
members, but the PROC entry gets reused for a new connection. It clearly
should not be a leader for that old decode group, but it may need to be
a leader for another group.
4) strange hunk in ProcKill
There seems to be some sort of merge/rebase issue, because this block of
code (line ~880) related to lock groups
/* Return PGPROC structure (and semaphore) to appropriate freelist */
proc->links.next = (SHM_QUEUE *) *procgloballist;
*procgloballist = proc;
got replaced by code relared to decode groups. That seems strange.
5) ReorderBufferCommitInternal
I see the LogicalLockTransaction() calls in ReorderBufferCommitInternal
have vastly variable comments. Some calls have no comment, some calls
have "obvious" comment like "Lock transaction before catalog access" and
one call has this very long comment
/*
* Output plugins can access catalog metadata and we
* do not have any control over that. We could ask
* them to call
* LogicalLockTransaction/LogicalUnlockTransaction
* APIs themselves, but that leads to unnecessary
* complications and expectations from plugin
* writers. We avoid this by calling these APIs
* here, thereby ensuring that the in-progress
* transaction will be around for the duration of
* the apply_change call below
*/
I find that rather inconsistent, and I'd say those comments are useless.
I suggest to remove all the per-call comments and instead add a comment
about the locking into the initial file-level comment, which already
explains handling of large transactions, etc.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, nikhils@2ndquadrant.com, david@pgmasters.net, andres@anarazel.de, andrew.dunstan@2ndquadrant.com, simon@2ndquadrant.com, craig@2ndquadrant.com, peter.eisentraut@2ndquadrant.com, petr.jelinek@2ndquadrant.com, y.sokolov@postgrespro.ru, s.kelvich@postgrespro.ru, 9erthalion6@gmail.com, tgl@sss.pgh.pa.us, sawada.mshk@gmail.com, robertmhaas@gmail.com
Subject: Re: [HACKERS] logical decoding of two-phase transactions
In-Reply-To: <0c7e9fb9-01f7-3284-da9e-22b6dde679d4@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox