Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1f1gFA-0001MT-2F for pgsql-hackers@arkaria.postgresql.org; Thu, 29 Mar 2018 22:42:04 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1f1gF8-0001ZJ-Ub for pgsql-hackers@arkaria.postgresql.org; Thu, 29 Mar 2018 22:42:02 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1f1gF8-0001Z9-N1 for pgsql-hackers@lists.postgresql.org; Thu, 29 Mar 2018 22:42:02 +0000 Received: from mail-wm0-x241.google.com ([2a00:1450:400c:c09::241]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1f1gF5-0001rB-0a for pgsql-hackers@postgresql.org; Thu, 29 Mar 2018 22:42:02 +0000 Received: by mail-wm0-x241.google.com with SMTP id x4so14072195wmh.5 for ; Thu, 29 Mar 2018 15:41:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=subject:from:to:cc:references:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=alQS+0gVTgovb1oCU/9lt8MvApP9HCYqyUBz14ccCWM=; b=oAR3S/ngv+OSSXP4i/0JrpDrD4TB255kxExFeJazIubXJfT2v+iKsxt/7lf8vjYbyW MCH1m2ACeFEUuykyBxvtD/8PzNfJkANw68fG8e8i5nDnoU2chCwWpVl4UiRD+2DyLl0r OOYoXqox7ml6nDf6P7BVvXPyXD/Wf6T9tuvFTUURaN9OmB5VhL+62aE8Y9h+4Ks8hEn6 NFsEeD1mAwRn3Y9ozMkgL8H2ft/1AX9u3MeYTdu5dMbfMXnV2X5qYN3WBgP+xDsqosUe 2o55B+xe5P9vlsfQ4qxnEPrCZNgxQtw0XR4X+mbWSSuTXUdjsCucCyhEg6sj8Zo+WBqW bCfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=alQS+0gVTgovb1oCU/9lt8MvApP9HCYqyUBz14ccCWM=; b=Ee2xet9UR83jKk6DPZgiiXbz70U+H1VDKyNCw1osu72MvebPXJ+RT/t0G5s12VVt43 qxDUG+owpCdFV8E1S/UtTQXXhs6ZUHAxUZSfO511GDT2VvFWXAqQf04oBRD2xdWtMY9X 2uWJe3VCgk2rBSjcEPQB3Vn26gZ8eOreRMMgnA7O8xi/UQthLFS+er0o76Cu2MAl3MMG z5GwSSNNubUnM53rpU3HAU8NRZejEiGY+c3PI4eYPm0kfY76D5ZOvNr2ZCAKbcYXGM4X pUIEUdWHxqr5bp+DsG+gBwnNWx/hsX7xptRwguB7T0gZzbevd4obf+g9PQ46BHUAYG1h QChg== X-Gm-Message-State: ALQs6tABQke9OKlZxZfAthYW4yxBtqSnTBExJ2GxVI0hvXHtzP/3lG1S NpcylbgfCAkzIDr6RHQL8T4BJQ== X-Google-Smtp-Source: AIpwx4+Jzs2xdPC87QkPdbT5o9nHvD7PTjo5nQlmZoP708mb2j+bzChe6ruQx9c0iZh30x5F/Wf+HQ== X-Received: by 10.28.211.147 with SMTP id k141mr495565wmg.15.1522363317908; Thu, 29 Mar 2018 15:41:57 -0700 (PDT) Received: from [192.168.0.106] (150.123.broadband11.iol.cz. [90.178.123.150]) by smtp.gmail.com with ESMTPSA id 90sm8278901wrl.10.2018.03.29.15.41.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 29 Mar 2018 15:41:56 -0700 (PDT) Subject: Re: [HACKERS] logical decoding of two-phase transactions From: Petr Jelinek To: Andres Freund , Tomas Vondra Cc: Nikhil Sontakke , Craig Ringer , Peter Eisentraut , Simon Riggs , Sokolov Yura , Stas Kelvich , Dmitry Dolgov <9erthalion6@gmail.com>, Tom Lane , Masahiko Sawada , PostgreSQL Hackers , Robert Haas References: <20180212162040.rsnlnysmf7n4yo7i@alap3.anarazel.de> <20180302005343.zv6nsqimcujzjrcd@alap3.anarazel.de> <20180329215815.36xxakgujx4luo4m@alap3.anarazel.de> Openpgp: preference=signencrypt Autocrypt: addr=petr.jelinek@2ndquadrant.com; keydata= xsFNBFawwSABEAC45Vua8rSggpM2kkMMmoOHz51a+fi3OXFfnrStVh80pM2/J9nSzjrIKQKf H5PtRnoYzSKnoUDoLOkLwDYitK3HzUUWXma+AWAqef2Y0GSZ7A3jJkD+JHOfkpS1CIpkEB/e WjA8Wqegwzc/OEE5111/l0Q1e6jxgZI86h8MMnm35Of1Y6Lz4kqWCyxHhaTJKTCkq66ICQ2C VJkbvuUoEE+Hr/I38dPHTAqIlCnycVnJPLHBQ/UYGHR/USF8qI/qJJx6tJcNAEqz69QZ/VlY 31UyJHdYe5NtBVz+rYZCOVWfpe3crbHuq9FjAPT+xcZefdxridmMKuW8eBmmXI9iZ3EG+X8J IsdLWL8QoqW0lnO1otYz1QDjdOvbEpq+u5/TAU8pgtaYH2J+6/GaQ6hMLyLBkaIAtnqlLCzq pxXcxMsFT14P5/rZA7EbBI8RS1ZIWJBlS2oO+dtVuENLJUC0SpptaRyn/uyOIopt7hSmBekL SzFnet7P9qofFJY+fQEz84v0G1gvwonHGHiCGgLpbMdPtnfPzda1aiTURvvOD7w/RmX1/ogk TqSGO4ejVNO/28QYouxmxzy8jbbIEiLCbHpLR/M/TvYHHPURxcx+MxVl3G+d/4Ihy4abxOdW pe9zLIcVMvqiUFQO28/79Gfhb7G4K+L21ORB82QYkzm3M7ebXQARAQABzSNQZXRyIEplbMOt bmVrIDxwam1vZG9zQHBqbW9kb3MubmV0PsLBgAQTAQoAKgIbAwUJBaOagAULCQgHAwUVCgkI CwUWAgMBAAIeAQIXgAUCVrDBqwIZAQAKCRBci2l0giH4y/QJEACfiLHyvxlwMvo3Y88Qa9QI i7Hu3ZW+xxojojdRd7do0nBnf2a4YstD9u6ZOLJx4f5Ijb/8LxY10NgLugmfQ7sw2W8ui6wT NiAeNHpr4wpT3IV+V53psOMRYPdq2zNTLO4iNxqPFPgNufR/m4FqcGe0Wo7U0wd1p0MErXTg EXJDVmjPDkYnzE9ejSCOxF57+o3xs6+eW9Al2m3nqP8JCzpauCr0gqpQzCaw0VZ05aLL0t9E B0fdm90+ydVe10Wg+I348005smuSSckBCHZqUfjjWvjwE6v/6dZ3/50ycAbi6cTe7Fo4kOSO sXBFXkzYGWYrEFPkzK0zDH65D98gL1+l9uokCR2t/OYVwudbUCV3cS3xRd0eFgHbXuzV+aZc jRjqkPfpCA2loKtrX2U739mpVSHnpIfSBf48fLg7O5dVn5ZCrydGkEzgvyynPo7IikQrIA02 91IoSp4RisKsJpaXHwHu6DH2rDCbYFumAMgXk2osLFX6VMPoSWKVRkr87A0VLafCtwaqr6Hy sFE6Lmu7jP4/BDr+9A2MXK+v0yYrXqqsZN6V9856ywlZrWK3MB6jc2bKpKS2YshneKxpitNV 0ITs2Li0OlAXOEHE0eXwI93POgLoyH+8JiNbem3orw3Mnc+7XpIY86AejwUiYl8T7EO/2tIx HDaXLK3BPNOdH87BTQRWsMEgARAA6Z3NrDLdcFF/P3TD/F7WQtMALEEKn5TXSF/K2VHt21GD 8tGY1DZhbxjLuQJeheY/2L4F4Y5WCkAoUdBXQs7nIauN+63EL88/qPdH6f3+/HZ+cmPG1E5o dBABCkC42eGcI5O8/9TAaFQcSzg86pdAE8v927tnc6aH4W4/Mj7RJNlQiZaz2ngCC9mr5uu+ CysOTfKCS0llNUkExZOJzZsQU3zDkcg3+xmgtHkCjglJ006KuytHj55HR6TzaKaAW8coHGoS OI/BQ9KAzvlj+32/zS8FSISZSTY3spMpraADsYklejCeJOmqQ3nTrCtwlX3hTSggfr4eelKA hdZKs1rIimNBtX90jJc0u2A/GmC959aTSrCUcfaPRweKUuexg7jxuhXyg9br2GscD2G6W7EV c+3s4dOHgoJFVMBBIqTtZviaNJ63IsZIFTYJzGKKMHOioZ3naoBVav0w63waDzj82nPYMIPj TxnClKiecuBQQPzMaG0jTmK0lis6rO0nIXrCwca8e/3Xzvi7sykOdcz9t4ef8HrDCCkG0yDO o8qHy1e7EtmJD7kWffIXZMt4TZVxpiOqgWtv+heJAfmqzr8pkNLUubmgvNdnakXJOJT41Lui 50C5OhpU9mfFCIDOeZXTQLsHA4KYaL8W/qcjv/YDmvpJtnK24y6tByWuTu7r/SMAEQEAAcLB ZQQYAQoADwUCVrDBIAIbDAUJBaOagAAKCRBci2l0giH4yyWpD/91jd4BMN2CDYqLtU7kaoAa aXnRVpf0z+R8rJ4LZJX1UNbHjsdClMQLfdBuZeDsdYar4iMfwScz6fiCiRwaRpr3XT6znaM0 wba6fjzEqVSHr3WB1jrb/TMiNaNgQJaRPhI5C7VsLuPrv5/L8gFmyjl712q7F3busGFBKrjK qTESMu+sHNvVeC3QyBFCe2fudbAJdaaRsUoTbAZYiowVoQHOow8eBoWY+RzQoYGuw1fls5Ck 85N73pYHXzjfk1s49sDKTYI263h9wbGGA234biHRI3NQN6sSYWRr/u5GOwTm/tzyadRj0pnZ S6KyLSEL4qx8jiyniD/XLoftlvR4YWQ0BXveEElx2n3TThLL1cVtOI4qE9BZKm32euBqwLta k9Jc659hmZB9E9aWLo8KcLjngwqSZupoHRPZ3yAlgZkpDoTIZpMCGxGEuuxgnosRbNpOJ1fE nNXenYYfi87tHRUCRKeHhoKkfGZEgC9zv9Y97j75npqV2tQyh4fNQT1tGdBigtvzI4nHM1/1 PLJtJBdRHucP76vUH9RlP9EE/Sg5DFQqsfSYPqxLW9r+ZfSEeT0APdz7TQC9MzNzUhyXvA5a RQQCRMXKbGaYhKkYEYiYOGqtTxN+PmyvNUBmFEcagBX2TXqEkgnI6zOnN13mB4SihJ2lne2r 9RNiPYs/hmRxYg== Message-ID: <87d3e688-89fa-bd76-8841-4fc7d16bca77@2ndquadrant.com> Date: Fri, 30 Mar 2018 00:41:56 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 30/03/18 00:30, Petr Jelinek wrote: > On 29/03/18 23:58, Andres Freund wrote: >> On 2018-03-29 23:52:18 +0200, Tomas Vondra wrote: >>>> I have added details about this in src/backend/storage/lmgr/README as >>>> suggested by you. >>>> >>> >>> Thanks. I think the README is a good start, but I think we also need to >>> improve the comments, which is usually more detailed than the README. >>> For example, it's not quite acceptable that LogicalLockTransaction and >>> LogicalUnlockTransaction have about no comments, especially when it's >>> meant to be public API for decoding plugins. >> >> FWIW, for me that's ground to not accept the feature. Burdening output >> plugins with this will make their development painful (because they'll >> have to adapt regularly) and correctness doubful (there's nothing >> checking for the lock being skipped). Another way needs to be found. >> > > I have to agree with Andres here. It's also visible in the latter > patches. The pgoutput patch forgets to call these new APIs completely. > The test_decoding calls them, but it does so even when it's processing > changes for committed transaction.. I think that should be avoided as it > means potentially doing SLRU lookup for every change. So doing it right > is indeed not easy. Ah turns out it actually does not need SLRU lookup in this case (I missed the reorder buffer call), so I take that part back. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services