Subject: Re: [HACKERS] logical decoding of two-phase transactions
To: Craig Ringer <craig@2ndquadrant.com>,
 Nikhil Sontakke <nikhils@2ndquadrant.com>
Cc: Sokolov Yura <y.sokolov@postgrespro.ru>,
 Stas Kelvich <s.kelvich@postgrespro.ru>, Andres Freund <andres@anarazel.de>,
 Dmitry Dolgov <9erthalion6@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>,
 Masahiko Sawada <sawada.mshk@gmail.com>, Simon Riggs
 <simon@2ndquadrant.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>,
 Robert Haas <robertmhaas@gmail.com>
References: <A434797D-1283-42C5-8CE5-7F1CD0B68F36@postgrespro.ru>
 <CAD21AoDDC7USpUyUim=7BwuFByauRGfLzkE7hhiV=jjDckc+KA@mail.gmail.com>
 <9CD693AF-51BE-4A19-9D26-16031CDAD9C9@postgrespro.ru>
 <20170404173133.olo7keeh3dzhriay@alap3.anarazel.de>
 <CA+q6zcUJLTwammygSSp0=evX_Ns46b2JkvWu214jbM-15xc_MA@mail.gmail.com>
 <19307.1494706954@sss.pgh.pa.us>
 <CA+q6zcX7m3aB5ziVA4D8oGRbtMJcaKXKZO6SEWVZ3hOZv7mQWQ@mail.gmail.com>
 <CAMGcDxfBkghqK94VZoOBE4XaOA9xZmkaM-opKeEzOrPmajVHBA@mail.gmail.com>
 <D6F3575D-BA17-43AB-A75F-12934885BADB@postgrespro.ru>
 <cd6a59942f42c37adc19eae23dfe701c@postgrespro.ru>
 <0e46521cbbca66c3298d1a9d672c9af0@postgrespro.ru>
 <CAMsr+YF4QfAKzsfHQ7MS+5NG3aY-NG3T_DdhcdPEPkWD4fo7fQ@mail.gmail.com>
 <CAMGcDxf83P5SGnGH52=_0wRP9pO6uRWCMRwAA0nxKtZvir2_vQ@mail.gmail.com>
 <CAMsr+YGX1z5t_MccrJxd4VJPgZKx17EsAQcAQ_1iJ4dKDp0LOg@mail.gmail.com>
 <CAMGcDxeDd4uB+tvaYrtjLgykeOpa-5nwPetNM+prCQ=wpPmPaQ@mail.gmail.com>
 <CAMsr+YH_x7Lrz3arUewzAk4=FZ-+hdum-eW7gZJBAEmzo937eg@mail.gmail.com>
From: Petr Jelinek <petr.jelinek@2ndquadrant.com>
Message-ID: <e55b1240-df0b-c5eb-bcdb-18902cdf42f8@2ndquadrant.com>
Date: Thu, 30 Nov 2017 00:40:36 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.4.0
MIME-Version: 1.0
In-Reply-To: 
 <CAMsr+YH_x7Lrz3arUewzAk4=FZ-+hdum-eW7gZJBAEmzo937eg@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk

Hi,

On 24/11/17 07:41, Craig Ringer wrote:
> On 24 November 2017 at 13:44, Nikhil Sontakke <nikhils@2ndquadrant.com
>  
> 
>     > How practical is adding a lock class?
> 
>     Am open to suggestions. This looks like it could work decently.
> 
> 
> It looks amazingly simple from here. Which probably means there's more
> to it that I haven't seen yet. I could use advice from someone who knows
> the locking subsystem better.
>  

Hmm, I don't like the interaction that would have with ROLLBACK, meaning
that ROLLBACK has to wait for decoding to finish which may take longer
than the transaction took itself (given potential network calls, it's
practically unbounded time).

I also think that if we'll want to add streaming of transactions in the
future, we'll face similar problem and the locking approach will not
work there as the transaction may still be locked by the owning backend
while we are decoding it.

From my perspective this patch changes the assumption in
HeapTupleSatisfiesVacuum() that changes done by aborted transaction
can't be seen by anybody else. That's clearly not true here as the
decoding can see it. So perhaps better approach would be to not return
HEAPTUPLE_DEAD if the transaction id is newer than the OldestXmin (same
logic we use for deleted tuples of committed transactions) in the
HeapTupleSatisfiesVacuum() even for aborted transactions. I also briefly
checked HOT pruning and AFAICS the normal HOT pruning (the one not
called by vacuum) also uses the xmin as authoritative even for aborted
txes so nothing needs to be done there probably.

In case we are worried that this affects cleanups of for example large
aborted COPY transactions and we think it's worth worrying about then we
could limit the new OldestXmin based logic only to catalog tuples as
those are the only ones we need available in decoding.

-- 
  Petr Jelinek                  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services