Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gRpFp-0001ji-Kv for pgsql-hackers@arkaria.postgresql.org; Wed, 28 Nov 2018 02:07:05 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gRpFo-0005fX-DG for pgsql-hackers@arkaria.postgresql.org; Wed, 28 Nov 2018 02:07:04 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gRpFo-0005fQ-1D for pgsql-hackers@lists.postgresql.org; Wed, 28 Nov 2018 02:07:04 +0000 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gRpFl-00086Z-Ap for pgsql-hackers@postgresql.org; Wed, 28 Nov 2018 02:07:03 +0000 Received: by mail-wm1-x343.google.com with SMTP id z18so922784wmc.4 for ; Tue, 27 Nov 2018 18:07:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=HBj2zqKnn8vbNNVu4BbEe/aw/MK6QAB0/gj1Mvxw6vo=; b=StKH97gGeLZYffOoTPJu3iBseT5stzfr3RV1g7QQYmewgmNWQLzARSGPq978B8jLV9 tJxKWnXMCstu9I2BV+xlnbHDmEisLKNaBl+ogrUisyiYn+xvHK5GD0OvU65CnfKjYbBV S8hEWfTljaYG467OPFR+13QRwoqY7rGZdH1WZFpapxo9he/s71i66fGdludVuKkkr2n6 iMRU17PhDFSzSNikR/lVIr2ZUOR8opHqGo4n9Bnhn/ZgMJCYWcCC0xABRu7qu1e6HLWZ g2icWO8xKKcXhan1h4BcSqet15v3t6pYiH9TBGoBSrILRRF5R+pKi+xm9xTTCl7yU/G1 xpAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=HBj2zqKnn8vbNNVu4BbEe/aw/MK6QAB0/gj1Mvxw6vo=; b=ZQDbKzfYMEVVXH4x1DIO4zAac3ZZmUP0LPUEzmaqwkGQnUv4JzJlQJzpdYBza/M0tD WrtQ8DGryryBryyaS0O0JNlgAKeGfuXQhUH6f3Yjx7rwxwHI1SvBZcxuFQffhx3vVQDN qSVlpE26I0VL3c4Eh6auJO+Rpca0t88+EsvnxeNec+N9XZCDSbG92gLWulzQ7QT8cGo5 SBN0ts51OWLGDHFnvhQE/OQhgS/JtwxeL3YRrDGunyCyq/erbBgZf/FS20aKEbUDHhGM BAkzYctopThSTD9B7Jz+tFaiY9f1t6MSv2/cbOcuAw5tlHyn/JST+PSczfrMj06lBkas fhrA== X-Gm-Message-State: AA+aEWa09WHskte7uQ7JxUkUM6/zverdoHWlcx35Ej8jSwQiGs61an4x BBRcBspFepQbAWy8xmgHX1yGsj0BoKf8ZX8acfYH4kjGNFh9RGYhnxs/VrAu2mTZHzRJDFXbfGV lsanuXFeIOCLT/Rfs3ZlididIMFZaZhF4AE0CLqgX7t3g72I+JoTIULmhazHH1rt7gFOBKpnAft VoHM68bLGkfhPXnw== X-Google-Smtp-Source: AFSGD/UopUfVN0mwnW9EDTpQ0Rg111Kc7wjb+QOdX7I9IP+xt8e81S/3wNHWD3YW/1v3v34WbTKmfg== X-Received: by 2002:a1c:9085:: with SMTP id s127mr955478wmd.114.1543370819566; Tue, 27 Nov 2018 18:06:59 -0800 (PST) Received: from [192.168.0.99] (150.123.broadband11.iol.cz. [90.178.123.150]) by smtp.gmail.com with ESMTPSA id t5sm955004wmd.15.2018.11.27.18.06.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 18:06:58 -0800 (PST) Subject: Re: logical decoding vs. VACUUM FULL / CLUSTER on table with TOAST-ed data To: Andres Freund , Tomas Vondra Cc: Masahiko Sawada , PostgreSQL-development References: <1a17c643-e9af-3dba-486b-fbe31bc1823a@2ndquadrant.com> <9d8e9a45-11e6-740a-b387-83c1c0829263@2ndquadrant.com> <7a4f2548-d574-c9e4-620f-cae4f00ffb45@2ndquadrant.com> <530c9399-14b5-2b50-02e3-d3f438578672@2ndquadrant.com> <20181128011439.7ob4i23yyaq5dn4j@alap3.anarazel.de> From: Petr Jelinek Openpgp: preference=signencrypt Autocrypt: addr=petr.jelinek@2ndquadrant.com; prefer-encrypt=mutual; keydata= xsFNBFawwSABEAC45Vua8rSggpM2kkMMmoOHz51a+fi3OXFfnrStVh80pM2/J9nSzjrIKQKf H5PtRnoYzSKnoUDoLOkLwDYitK3HzUUWXma+AWAqef2Y0GSZ7A3jJkD+JHOfkpS1CIpkEB/e WjA8Wqegwzc/OEE5111/l0Q1e6jxgZI86h8MMnm35Of1Y6Lz4kqWCyxHhaTJKTCkq66ICQ2C VJkbvuUoEE+Hr/I38dPHTAqIlCnycVnJPLHBQ/UYGHR/USF8qI/qJJx6tJcNAEqz69QZ/VlY 31UyJHdYe5NtBVz+rYZCOVWfpe3crbHuq9FjAPT+xcZefdxridmMKuW8eBmmXI9iZ3EG+X8J IsdLWL8QoqW0lnO1otYz1QDjdOvbEpq+u5/TAU8pgtaYH2J+6/GaQ6hMLyLBkaIAtnqlLCzq pxXcxMsFT14P5/rZA7EbBI8RS1ZIWJBlS2oO+dtVuENLJUC0SpptaRyn/uyOIopt7hSmBekL SzFnet7P9qofFJY+fQEz84v0G1gvwonHGHiCGgLpbMdPtnfPzda1aiTURvvOD7w/RmX1/ogk TqSGO4ejVNO/28QYouxmxzy8jbbIEiLCbHpLR/M/TvYHHPURxcx+MxVl3G+d/4Ihy4abxOdW pe9zLIcVMvqiUFQO28/79Gfhb7G4K+L21ORB82QYkzm3M7ebXQARAQABzSNQZXRyIEplbMOt bmVrIDxwam1vZG9zQHBqbW9kb3MubmV0PsLBgAQTAQoAKgIbAwUJBaOagAULCQgHAwUVCgkI CwUWAgMBAAIeAQIXgAUCVrDBqwIZAQAKCRBci2l0giH4y/QJEACfiLHyvxlwMvo3Y88Qa9QI i7Hu3ZW+xxojojdRd7do0nBnf2a4YstD9u6ZOLJx4f5Ijb/8LxY10NgLugmfQ7sw2W8ui6wT NiAeNHpr4wpT3IV+V53psOMRYPdq2zNTLO4iNxqPFPgNufR/m4FqcGe0Wo7U0wd1p0MErXTg EXJDVmjPDkYnzE9ejSCOxF57+o3xs6+eW9Al2m3nqP8JCzpauCr0gqpQzCaw0VZ05aLL0t9E B0fdm90+ydVe10Wg+I348005smuSSckBCHZqUfjjWvjwE6v/6dZ3/50ycAbi6cTe7Fo4kOSO sXBFXkzYGWYrEFPkzK0zDH65D98gL1+l9uokCR2t/OYVwudbUCV3cS3xRd0eFgHbXuzV+aZc jRjqkPfpCA2loKtrX2U739mpVSHnpIfSBf48fLg7O5dVn5ZCrydGkEzgvyynPo7IikQrIA02 91IoSp4RisKsJpaXHwHu6DH2rDCbYFumAMgXk2osLFX6VMPoSWKVRkr87A0VLafCtwaqr6Hy sFE6Lmu7jP4/BDr+9A2MXK+v0yYrXqqsZN6V9856ywlZrWK3MB6jc2bKpKS2YshneKxpitNV 0ITs2Li0OlAXOEHE0eXwI93POgLoyH+8JiNbem3orw3Mnc+7XpIY86AejwUiYl8T7EO/2tIx HDaXLK3BPNOdH87BTQRWsMEgARAA6Z3NrDLdcFF/P3TD/F7WQtMALEEKn5TXSF/K2VHt21GD 8tGY1DZhbxjLuQJeheY/2L4F4Y5WCkAoUdBXQs7nIauN+63EL88/qPdH6f3+/HZ+cmPG1E5o dBABCkC42eGcI5O8/9TAaFQcSzg86pdAE8v927tnc6aH4W4/Mj7RJNlQiZaz2ngCC9mr5uu+ CysOTfKCS0llNUkExZOJzZsQU3zDkcg3+xmgtHkCjglJ006KuytHj55HR6TzaKaAW8coHGoS OI/BQ9KAzvlj+32/zS8FSISZSTY3spMpraADsYklejCeJOmqQ3nTrCtwlX3hTSggfr4eelKA hdZKs1rIimNBtX90jJc0u2A/GmC959aTSrCUcfaPRweKUuexg7jxuhXyg9br2GscD2G6W7EV c+3s4dOHgoJFVMBBIqTtZviaNJ63IsZIFTYJzGKKMHOioZ3naoBVav0w63waDzj82nPYMIPj TxnClKiecuBQQPzMaG0jTmK0lis6rO0nIXrCwca8e/3Xzvi7sykOdcz9t4ef8HrDCCkG0yDO o8qHy1e7EtmJD7kWffIXZMt4TZVxpiOqgWtv+heJAfmqzr8pkNLUubmgvNdnakXJOJT41Lui 50C5OhpU9mfFCIDOeZXTQLsHA4KYaL8W/qcjv/YDmvpJtnK24y6tByWuTu7r/SMAEQEAAcLB ZQQYAQoADwUCVrDBIAIbDAUJBaOagAAKCRBci2l0giH4yyWpD/91jd4BMN2CDYqLtU7kaoAa aXnRVpf0z+R8rJ4LZJX1UNbHjsdClMQLfdBuZeDsdYar4iMfwScz6fiCiRwaRpr3XT6znaM0 wba6fjzEqVSHr3WB1jrb/TMiNaNgQJaRPhI5C7VsLuPrv5/L8gFmyjl712q7F3busGFBKrjK qTESMu+sHNvVeC3QyBFCe2fudbAJdaaRsUoTbAZYiowVoQHOow8eBoWY+RzQoYGuw1fls5Ck 85N73pYHXzjfk1s49sDKTYI263h9wbGGA234biHRI3NQN6sSYWRr/u5GOwTm/tzyadRj0pnZ S6KyLSEL4qx8jiyniD/XLoftlvR4YWQ0BXveEElx2n3TThLL1cVtOI4qE9BZKm32euBqwLta k9Jc659hmZB9E9aWLo8KcLjngwqSZupoHRPZ3yAlgZkpDoTIZpMCGxGEuuxgnosRbNpOJ1fE nNXenYYfi87tHRUCRKeHhoKkfGZEgC9zv9Y97j75npqV2tQyh4fNQT1tGdBigtvzI4nHM1/1 PLJtJBdRHucP76vUH9RlP9EE/Sg5DFQqsfSYPqxLW9r+ZfSEeT0APdz7TQC9MzNzUhyXvA5a RQQCRMXKbGaYhKkYEYiYOGqtTxN+PmyvNUBmFEcagBX2TXqEkgnI6zOnN13mB4SihJ2lne2r 9RNiPYs/hmRxYg== Message-ID: Date: Wed, 28 Nov 2018 03:06:58 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <20181128011439.7ob4i23yyaq5dn4j@alap3.anarazel.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk Hi, On 28/11/2018 02:14, Andres Freund wrote: > Hi, > > On 2018-11-28 02:04:18 +0100, Tomas Vondra wrote: >> >> On 11/24/18 12:20 AM, Tomas Vondra wrote: >>> ... >>> >>> OK, here's an updated patch, tweaking the reorderbuffer part. I plan >>> to push this sometime mid next week. >>> >> >> Pushed and backpatched to 9.4- (same as e9edc1ba). > > Backpatching seems on the more aggressive end of things for an > optimization. Could you at least announce that beforehand next time? > Well, it may be optimization, but from what I've seen the problems arising from this can easily prevent logical replication from working altogether as reorder buffer hits OOM on bigger tables. So ISTM that it does warrant backpatch. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services