Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oHK4Z-0003q1-3l for pgsql-bugs@arkaria.postgresql.org; Fri, 29 Jul 2022 07:06:11 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1oHK4X-0007xE-Ep for pgsql-bugs@arkaria.postgresql.org; Fri, 29 Jul 2022 07:06:09 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oHK4X-0007x5-04 for pgsql-bugs@lists.postgresql.org; Fri, 29 Jul 2022 07:06:09 +0000 Received: from mail-ed1-x536.google.com ([2a00:1450:4864:20::536]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1oHK4Q-0003zZ-In for pgsql-bugs@lists.postgresql.org; Fri, 29 Jul 2022 07:06:07 +0000 Received: by mail-ed1-x536.google.com with SMTP id i13so4703705edj.11 for ; Fri, 29 Jul 2022 00:06:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=LDdMJabxxzgzFLXyy5pn+IIZ5RsPJ23RBIJbQDwlidU=; b=fOeUaVDOp8YWbwuLrTntKQ5rY8P2pVPo2QU+L+kqmKcX+3ROPZD9C66XTt0XFGHAkL 2IGb0Tke7ZFaJAwERppbgvSI9mEb9+sr0BL2G5k2D6rSfDDF2hSRKQXfPFDLrgHGRilk qKlPPm8CnzwW67qkeLmfUtIWh1ZeHwlqhIwN4/c0dEiKCZzDBvVAY/bCyxgmrKYzvvw7 WkvQwGFLVjyIukcF6OLqeupSit6RnuDPRDC56sZHaEER6AyaxbaVVNyvujcS1nDRmGyV 1Kn496PozD2JeD6SmoQpV8ZquzUdF0sspOX+VJwd4yIIC8GUcSnDcxBqKBC3C3DgVM7g Pq/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LDdMJabxxzgzFLXyy5pn+IIZ5RsPJ23RBIJbQDwlidU=; b=FZSnOiyMO289RxmNfWLmd9hQhBQJlF4Dr1oJAp4yLpXcytFm2ZIRyKKeeNTHMedIvf ljieZ2XCs5L4iGz9JYgC/b4dn7L/vKd34uNSUSxXiVatyMSEAMYSIizXIirNDsn04KXc xLulgRn6wR46rz6+aR8xTN/kMITMhhteKV5QDbjqUEo50JGGqyerH4NCTEVrlmq360gi z7+3DldUpsmEcqjWEekLUfBdHOoC7SK6P0eHGDutvtzZeMIiG3h2s1fzU+mZzmbHI/KD sdT5lBrhuoJA8T2T+3LSRz+GfNuPz/mY1RZVpALE9wM7QWsW0Kw8FpXsiVoLSahY0/tB 9eyQ== X-Gm-Message-State: AJIora9DRgLLZ52gEPvL8M8PrvkgrQ4x/j8uxUiLmvQgTiiSR81QTMS4 O4PdB4tckFP9tDf2BwVpe8Py5K3HgZ+r/QlK+Fk= X-Google-Smtp-Source: AGRyM1s2osMPBzeVOtwkQkprE76/ll/6T3Q0VXynlJY22hWw6DDL/H0gLX6b+Sjc2dafTq3BUZRImyoGFPL32CSQMDI= X-Received: by 2002:a05:6402:5108:b0:43b:e395:d2fb with SMTP id m8-20020a056402510800b0043be395d2fbmr2274283edd.260.1659078359562; Fri, 29 Jul 2022 00:05:59 -0700 (PDT) MIME-Version: 1.0 References: <20220714115155.GA5439@depesz.com> <20220718094330.GA24063@depesz.com> In-Reply-To: From: Ajin Cherian Date: Fri, 29 Jul 2022 17:05:46 +1000 Message-ID: Subject: Re: Excessive number of replication slots for 12->14 logical replication To: Peter Smith Cc: "houzj.fnst@fujitsu.com" , Amit Kapila , Hubert Lubaczewski , PostgreSQL mailing lists Content-Type: multipart/mixed; boundary="00000000000046c73005e4ec4869" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --00000000000046c73005e4ec4869 Content-Type: text/plain; charset="UTF-8" On Thu, Jul 28, 2022 at 11:56 AM Peter Smith wrote: > > Here are some review comments for the v2-0001 patch: > > ====== > > 1. Commit message > > The commit message should give some reason why relocating the origin > slot drop code is expected to fix the reported problem. > > ====== > Updated. > 2. src/backend/replication/logical/tablesync.c > > + /* reset the origin session before dropping */ > + replorigin_session_reset(); > + > + replorigin_session_origin = InvalidRepOriginId; > + replorigin_session_origin_lsn = InvalidXLogRecPtr; > + replorigin_session_origin_timestamp = 0; > > 2a. > Uppercase comment Fixed. > > 2b. > IIUC you are not doing the reset because you particularly want to do a > reset, but really because this is the only (?) way to dis-associate > the current process from owning the slot. Otherwise, the slot would be > considered still "busy" and the subsequent replorigin_drop_by_name > would be rejected. I thought the current comment should be expanded to > explain all this background. Updated. > > 2c. > Perhaps it is non-standard to do so, but I wondered if you can just > call pg_replication_origin_session_reset instead of > replorigin_session_reset here so that there would only be 1 LOC > instead of 4. That is a psql function, invoking that indirectly is not really a standard practice. > > ~~~ > > 3. src/backend/replication/logical/tablesync.c > > + /* > * Cleanup the tablesync slot. > * > * This has to be done after updating the state because otherwise if > > Doesn't the same note ("This has to be done after...") also apply to > the code dropping the origin slot? Maybe this note needs to be either > duplicated or just put a common note about this above both of those > slot drops. > Updated. regards, Ajin Cherian Fujitsu Australia --00000000000046c73005e4ec4869 Content-Type: application/octet-stream; name="v3-0001-fix-excessive-replicating-origin-slots-issue.patch" Content-Disposition: attachment; filename="v3-0001-fix-excessive-replicating-origin-slots-issue.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_l664ecvl0 RnJvbSBjODUyNDkzNjBkOGYxYTQ1ZDU1YjNlYWIzZTk5NWIwNDM2YTBiOWIyIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBBamluIENoZXJpYW4gPGFqaW5jQGZhc3QuYXUuZnVqaXRzdS5j b20+CkRhdGU6IEZyaSwgMjkgSnVsIDIwMjIgMDI6MzE6NTEgLTA0MDAKU3ViamVjdDogW1BBVENI IHYzXSBmaXggZXhjZXNzaXZlIHJlcGxpY2F0aW5nIG9yaWdpbiBzbG90cyBpc3N1ZS4KCkluIHRo ZSBjdXJyZW50IChIRUFEKSBjb2RlIHRoZSBvcmlnaW4gc2xvdHMgb2YgdGhlIGNvbXBsZXRlZCB0 YWJsZXN5bmMgd29ya2VyCmFyZSBkcm9wcGVkIGJ5IHRoZSBhcHBseSB3b3JrZXIsIGJ1dCB0aGlz IG1lYW5zIHRoZXJlIGNhbiBiZSBhIHNtYWxsIGxhZwpiZXR3ZWVuIHdoZW4gdGhlIHRhYmxlc3lu YyB3b3JrZXIgZXhpdHMsIGFuZCB3aGVuIGl0cyBvcmlnaW4gc2xvdCBpcyBhY3R1YWxseSBkcm9w cGVkLgoKTWVhbndoaWxlIG5ldyB0YWJsZXN5bmMgd29ya2VycyBhcmUgbGF1bmNoZWQgYW5kIHRy eSBhbmQgYWNxdWlyZSBzbG90cyBiZWZvcmUgdGhlCnByZXZpb3VzbHkgZmluaXNoZWQgdGFibGVz eW5jIHdvcmtlciBzbG90cyBjb3VsZCBiZSBkcm9wcGVkLiBJZiB0aGlzIGhhcHBlbnMgdGhlCnRo ZSBhdmFpbGFibGUgbnVtYmVyIG9mIHNsb3RzIG1pZ2h0IGJlIGV4Y2VlZGVkIGxlYWRpbmcgdG8g dGhlIGVycm9yIHJlcG9ydGVkLgoKVG8gYXZvaWQgdGhpcyBsYWcsIHRoZSBkcm9wcGluZyBvZiBy ZXBsaWNhdGluZyBvcmlnaW4gc2xvdHMgaXMgbW92ZWQgdG8gdGhlCnRhYmxlc3luYyB3b3JrZXIg d2hlbiBpdCBleGl0cy4KLS0tCiBzcmMvYmFja2VuZC9yZXBsaWNhdGlvbi9sb2dpY2FsL3RhYmxl c3luYy5jIHwgNjEgKysrKysrKysrKysrKysrKystLS0tLS0tLS0tLS0KIDEgZmlsZSBjaGFuZ2Vk LCAzNiBpbnNlcnRpb25zKCspLCAyNSBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9zcmMvYmFj a2VuZC9yZXBsaWNhdGlvbi9sb2dpY2FsL3RhYmxlc3luYy5jIGIvc3JjL2JhY2tlbmQvcmVwbGlj YXRpb24vbG9naWNhbC90YWJsZXN5bmMuYwppbmRleCA2YTAxZmZkLi40MGRlNzUzIDEwMDY0NAot LS0gYS9zcmMvYmFja2VuZC9yZXBsaWNhdGlvbi9sb2dpY2FsL3RhYmxlc3luYy5jCisrKyBiL3Ny Yy9iYWNrZW5kL3JlcGxpY2F0aW9uL2xvZ2ljYWwvdGFibGVzeW5jLmMKQEAgLTI5MSw2ICsyOTEs NyBAQCBwcm9jZXNzX3N5bmNpbmdfdGFibGVzX2Zvcl9zeW5jKFhMb2dSZWNQdHIgY3VycmVudF9s c24pCiAJewogCQlUaW1lTGluZUlECXRsaTsKIAkJY2hhcgkJc3luY3Nsb3RuYW1lW05BTUVEQVRB TEVOXSA9IHswfTsKKwkJY2hhcgkJb3JpZ2lubmFtZVtOQU1FREFUQUxFTl0gPSB7MH07CiAKIAkJ TXlMb2dpY2FsUmVwV29ya2VyLT5yZWxzdGF0ZSA9IFNVQlJFTF9TVEFURV9TWU5DRE9ORTsKIAkJ TXlMb2dpY2FsUmVwV29ya2VyLT5yZWxzdGF0ZV9sc24gPSBjdXJyZW50X2xzbjsKQEAgLTMxNSwx MyArMzE2LDQzIEBAIHByb2Nlc3Nfc3luY2luZ190YWJsZXNfZm9yX3N5bmMoWExvZ1JlY1B0ciBj dXJyZW50X2xzbikKIAkJICovCiAJCXdhbHJjdl9lbmRzdHJlYW1pbmcoTG9nUmVwV29ya2VyV2Fs UmN2Q29ubiwgJnRsaSk7CiAKKyAgICAgICAgLyoKKyAgICAgICAgICogQ2xlYW51cCB0aGUgdGFi bGVzeW5jIHNsb3QgYW5kIHRoZSBvcmlnaW4gdHJhY2tpbmcgaWYgZXhpc3RzLgorICAgICAgICAg KgorICAgICAgICAgKiBUaGlzIGhhcyB0byBiZSBkb25lIGFmdGVyIHVwZGF0aW5nIHRoZSBzdGF0 ZSBiZWNhdXNlIG90aGVyd2lzZSBpZgorICAgICAgICAgKiB0aGVyZSBpcyBhbiBlcnJvciB3aGls ZSBkb2luZyB0aGUgZGF0YWJhc2Ugb3BlcmF0aW9ucyB3ZSB3b24ndCBiZQorICAgICAgICAgKiBh YmxlIHRvIHJvbGxiYWNrIGRyb3BwZWQgc2xvdCBvciBvcmlnaW4gdHJhY2tpbmcuCisgICAgICAg ICAqLworCiAJCS8qCi0JCSAqIENsZWFudXAgdGhlIHRhYmxlc3luYyBzbG90LgorCQkgKiBDbGVh bnVwIHRoZSB0YWJsZXN5bmMgb3JpZ2luIHRyYWNraW5nIGlmIGV4aXN0cy4KKwkJICovCisJCVJl cGxpY2F0aW9uT3JpZ2luTmFtZUZvclRhYmxlc3luYyhNeUxvZ2ljYWxSZXBXb3JrZXItPnN1Ymlk LAorCQkJCQkJCQkJCSAgTXlMb2dpY2FsUmVwV29ya2VyLT5yZWxpZCwKKwkJCQkJCQkJCQkgIG9y aWdpbm5hbWUsCisJCQkJCQkJCQkJICBzaXplb2Yob3JpZ2lubmFtZSkpOworCisJCS8qIAorCQkg KiBSZXNldCB0aGUgb3JpZ2luIHNlc3Npb24gYmVmb3JlIGRyb3BwaW5nLgogCQkgKgotCQkgKiBU aGlzIGhhcyB0byBiZSBkb25lIGFmdGVyIHVwZGF0aW5nIHRoZSBzdGF0ZSBiZWNhdXNlIG90aGVy d2lzZSBpZgotCQkgKiB0aGVyZSBpcyBhbiBlcnJvciB3aGlsZSBkb2luZyB0aGUgZGF0YWJhc2Ug b3BlcmF0aW9ucyB3ZSB3b24ndCBiZQotCQkgKiBhYmxlIHRvIHJvbGxiYWNrIGRyb3BwZWQgc2xv dC4KKwkJICogVGhpcyBpcyByZXF1aXJlZCB0byByZXNldCB0aGUgb3duZXJzaGlwIG9mIHRoZSBz bG90CisJCSAqIGFuZCBhbGxvdyB0aGUgc2xvdCB0byBiZSBkcm9wcGVkLgogCQkgKi8KKwkJcmVw bG9yaWdpbl9zZXNzaW9uX3Jlc2V0KCk7CisKKwkJcmVwbG9yaWdpbl9zZXNzaW9uX29yaWdpbiA9 IEludmFsaWRSZXBPcmlnaW5JZDsKKwkJcmVwbG9yaWdpbl9zZXNzaW9uX29yaWdpbl9sc24gPSBJ bnZhbGlkWExvZ1JlY1B0cjsKKwkJcmVwbG9yaWdpbl9zZXNzaW9uX29yaWdpbl90aW1lc3RhbXAg PSAwOworCisJCS8qCisJCSAqIFRoZXJlIGlzIGEgY2hhbmNlIHRoYXQgdGhlIHVzZXIgaXMgY29u Y3VycmVudGx5IHBlcmZvcm1pbmcKKwkJICogcmVmcmVzaCBmb3IgdGhlIHN1YnNjcmlwdGlvbiB3 aGVyZSB3ZSByZW1vdmUgdGhlIHRhYmxlCisJCSAqIHN0YXRlIGFuZCBpdHMgb3JpZ2luIGFuZCBi eSB0aGlzIHRpbWUgdGhlIG9yaWdpbiBtaWdodCBiZQorCQkgKiBhbHJlYWR5IHJlbW92ZWQuIFNv IHBhc3NpbmcgbWlzc2luZ19vayA9IHRydWUuCisJCSAqLworCQlyZXBsb3JpZ2luX2Ryb3BfYnlf bmFtZShvcmlnaW5uYW1lLCB0cnVlLCBmYWxzZSk7CisKKwkJLyogQ2xlYW51cCB0aGUgdGFibGVz eW5jIHNsb3QuICovCiAJCVJlcGxpY2F0aW9uU2xvdE5hbWVGb3JUYWJsZXN5bmMoTXlMb2dpY2Fs UmVwV29ya2VyLT5zdWJpZCwKIAkJCQkJCQkJCQlNeUxvZ2ljYWxSZXBXb3JrZXItPnJlbGlkLAog CQkJCQkJCQkJCXN5bmNzbG90bmFtZSwKQEAgLTQ0MSw4ICs0NzIsNiBAQCBwcm9jZXNzX3N5bmNp bmdfdGFibGVzX2Zvcl9hcHBseShYTG9nUmVjUHRyIGN1cnJlbnRfbHNuKQogCQkJICovCiAJCQlp ZiAoY3VycmVudF9sc24gPj0gcnN0YXRlLT5sc24pCiAJCQl7Ci0JCQkJY2hhcgkJb3JpZ2lubmFt ZVtOQU1FREFUQUxFTl07Ci0KIAkJCQlyc3RhdGUtPnN0YXRlID0gU1VCUkVMX1NUQVRFX1JFQURZ OwogCQkJCXJzdGF0ZS0+bHNuID0gY3VycmVudF9sc247CiAJCQkJaWYgKCFzdGFydGVkX3R4KQpA QCAtNDUxLDI3ICs0ODAsOSBAQCBwcm9jZXNzX3N5bmNpbmdfdGFibGVzX2Zvcl9hcHBseShYTG9n UmVjUHRyIGN1cnJlbnRfbHNuKQogCQkJCQlzdGFydGVkX3R4ID0gdHJ1ZTsKIAkJCQl9CiAKLQkJ CQkvKgotCQkJCSAqIFJlbW92ZSB0aGUgdGFibGVzeW5jIG9yaWdpbiB0cmFja2luZyBpZiBleGlz dHMuCi0JCQkJICoKLQkJCQkgKiBUaGUgbm9ybWFsIGNhc2Ugb3JpZ2luIGRyb3AgaXMgZG9uZSBo ZXJlIGluc3RlYWQgb2YgaW4gdGhlCi0JCQkJICogcHJvY2Vzc19zeW5jaW5nX3RhYmxlc19mb3Jf c3luYyBmdW5jdGlvbiBiZWNhdXNlIHdlIGRvbid0Ci0JCQkJICogYWxsb3cgdG8gZHJvcCB0aGUg b3JpZ2luIHRpbGwgdGhlIHByb2Nlc3Mgb3duaW5nIHRoZSBvcmlnaW4KLQkJCQkgKiBpcyBhbGl2 ZS4KLQkJCQkgKgotCQkJCSAqIFRoZXJlIGlzIGEgY2hhbmNlIHRoYXQgdGhlIHVzZXIgaXMgY29u Y3VycmVudGx5IHBlcmZvcm1pbmcKLQkJCQkgKiByZWZyZXNoIGZvciB0aGUgc3Vic2NyaXB0aW9u IHdoZXJlIHdlIHJlbW92ZSB0aGUgdGFibGUKLQkJCQkgKiBzdGF0ZSBhbmQgaXRzIG9yaWdpbiBh bmQgYnkgdGhpcyB0aW1lIHRoZSBvcmlnaW4gbWlnaHQgYmUKLQkJCQkgKiBhbHJlYWR5IHJlbW92 ZWQuIFNvIHBhc3NpbmcgbWlzc2luZ19vayA9IHRydWUuCi0JCQkJICovCi0JCQkJUmVwbGljYXRp b25PcmlnaW5OYW1lRm9yVGFibGVzeW5jKE15TG9naWNhbFJlcFdvcmtlci0+c3ViaWQsCi0JCQkJ CQkJCQkJCQkgIHJzdGF0ZS0+cmVsaWQsCi0JCQkJCQkJCQkJCQkgIG9yaWdpbm5hbWUsCi0JCQkJ CQkJCQkJCQkgIHNpemVvZihvcmlnaW5uYW1lKSk7Ci0JCQkJcmVwbG9yaWdpbl9kcm9wX2J5X25h bWUob3JpZ2lubmFtZSwgdHJ1ZSwgZmFsc2UpOwogCiAJCQkJLyoKLQkJCQkgKiBVcGRhdGUgdGhl IHN0YXRlIHRvIFJFQURZIG9ubHkgYWZ0ZXIgdGhlIG9yaWdpbiBjbGVhbnVwLgorCQkJCSAqIFVw ZGF0ZSB0aGUgc3RhdGUgdG8gUkVBRFkuCiAJCQkJICovCiAJCQkJVXBkYXRlU3Vic2NyaXB0aW9u UmVsU3RhdGUoTXlMb2dpY2FsUmVwV29ya2VyLT5zdWJpZCwKIAkJCQkJCQkJCQkgICByc3RhdGUt PnJlbGlkLCByc3RhdGUtPnN0YXRlLAotLSAKMS44LjMuMQoK --00000000000046c73005e4ec4869--