Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lMvZi-00026l-2F for pgsql-bugs@arkaria.postgresql.org; Thu, 18 Mar 2021 16:32:42 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lMvZh-0003H5-0Z for pgsql-bugs@arkaria.postgresql.org; Thu, 18 Mar 2021 16:32:41 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lMvZg-0003Gy-Qv for pgsql-bugs@lists.postgresql.org; Thu, 18 Mar 2021 16:32:40 +0000 Received: from oss.nttdata.com ([49.212.34.109]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lMvZb-0000dN-88 for pgsql-bugs@lists.postgresql.org; Thu, 18 Mar 2021 16:32:39 +0000 Received: from hnk.local (p2350207-ipbf2126funabasi.chiba.ocn.ne.jp [114.167.184.207]) by oss.nttdata.com (Postfix) with ESMTPSA id 69D6B62800; Fri, 19 Mar 2021 01:32:30 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.102.3 at oss.nttdata.com Subject: Re: BUG #16722: PG hanging on COPY when table has close to 2^32 toasts in the table. To: tomohiro hiramitsu Cc: Tom Lane , Kyotaro Horiguchi , Kasahara Tatsuhito , Magnus Hagander , Andres Freund , skoposov@ed.ac.uk, PostgreSQL mailing lists References: <20210301.112452.1902584192027811521.horikyota.ntt@gmail.com> <666801.1614567656@sss.pgh.pa.us> From: Fujii Masao Message-ID: Date: Fri, 19 Mar 2021 01:32:29 +0900 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------54926145FE0A55F768D1D710" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------54926145FE0A55F768D1D710 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 2021/03/17 14:30, tomohiro hiramitsu wrote: > v4-0001-GetNewOidWithIndex_log_output.patch Thanks for updating the patch! I applied the following and cosmetic changes to the patch. Attached is the updated version of the patch. + ereport(LOG, + (errmsg("still finding an unused OID within relation \"%s\"", + ereport(LOG, + (errmsg("the new OID has been assigned in relation \"%s\" after \"%llu\" retries", WARNING is more suitable for these kinds of messages? I changed the log level here to WARNING. +#define GETNEWOID_NOTIFY_MINVAL 1000000 +#define GETNEWOID_NOTIFY_MAXVAL 100000000 If we use WARNING, GETNEWOID_WARN_THRESHOLD and GETNEWOID_WARN_MAX_INTERVAL sound better to me for the meanings of them. I renamed them. + if (retry_count >= GETNEWOID_NOTIFY_MINVAL) ">=" should be ">"? Otherwise, the message for completion of OID assignment can be logged even when no message "still finding ..." is output. I fixed this. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION --------------54926145FE0A55F768D1D710 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="v5-0001-GetNewOidWithIndex_log_output.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="v5-0001-GetNewOidWithIndex_log_output.patch" ZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL2NhdGFsb2cvY2F0YWxvZy5jIGIvc3JjL2JhY2tl bmQvY2F0YWxvZy9jYXRhbG9nLmMKaW5kZXggZTJlZDgwYTVkZS4uYTkwYWIzNjE0YyAxMDA2 NDQKLS0tIGEvc3JjL2JhY2tlbmQvY2F0YWxvZy9jYXRhbG9nLmMKKysrIGIvc3JjL2JhY2tl bmQvY2F0YWxvZy9jYXRhbG9nLmMKQEAgLTQ3LDYgKzQ3LDEzIEBACiAjaW5jbHVkZSAidXRp bHMvc25hcG1nci5oIgogI2luY2x1ZGUgInV0aWxzL3N5c2NhY2hlLmgiCiAKKy8qCisgKiBQ YXJhbWV0ZXJzIHRvIGRldGVybWluZSB3aGVuIHRvIGVtaXQgYSB3YXJuaW5nIGluCisgKiBH ZXROZXdPaWRXaXRoSW5kZXgoKQorICovCisjZGVmaW5lIEdFVE5FV09JRF9XQVJOX1RIUkVT SE9MRCAxMDAwMDAwCisjZGVmaW5lIEdFVE5FV09JRF9XQVJOX01BWF9JTlRFUlZBTCAxMDAw MDAwMDAKKwogLyoKICAqIElzU3lzdGVtUmVsYXRpb24KICAqCQlUcnVlIGlmZiB0aGUgcmVs YXRpb24gaXMgZWl0aGVyIGEgc3lzdGVtIGNhdGFsb2cgb3IgYSB0b2FzdCB0YWJsZS4KQEAg LTMxOCw2ICszMjUsOCBAQCBHZXROZXdPaWRXaXRoSW5kZXgoUmVsYXRpb24gcmVsYXRpb24s IE9pZCBpbmRleElkLCBBdHRyTnVtYmVyIG9pZGNvbHVtbikKIAlTeXNTY2FuRGVzYyBzY2Fu OwogCVNjYW5LZXlEYXRhIGtleTsKIAlib29sCQljb2xsaWRlczsKKwl1aW50NjQJCXJldHJp ZXMgPSAwOworCXVpbnQ2NAkJcmV0cmllc19iZWZvcmVfd2FybiA9IEdFVE5FV09JRF9XQVJO X1RIUkVTSE9MRDsKIAogCS8qIE9ubHkgc3lzdGVtIHJlbGF0aW9ucyBhcmUgc3VwcG9ydGVk ICovCiAJQXNzZXJ0KElzU3lzdGVtUmVsYXRpb24ocmVsYXRpb24pKTsKQEAgLTM1Myw4ICsz NjIsNDggQEAgR2V0TmV3T2lkV2l0aEluZGV4KFJlbGF0aW9uIHJlbGF0aW9uLCBPaWQgaW5k ZXhJZCwgQXR0ck51bWJlciBvaWRjb2x1bW4pCiAJCWNvbGxpZGVzID0gSGVhcFR1cGxlSXNW YWxpZChzeXN0YWJsZV9nZXRuZXh0KHNjYW4pKTsKIAogCQlzeXN0YWJsZV9lbmRzY2FuKHNj YW4pOworCisJCS8qCisJCSAqIFdhcm4gdXNlcnMgdGhhdCB3ZSBpdGVyYXRlIG1vcmUgdGhh biBHRVRORVdPSURfV0FSTl9USFJFU0hPTEQgYnV0CisJCSAqIGhhdmUgbm90IHlldCBmb3Vu ZCBPSUQgdW51c2VkIGluIHRoZSByZWxhdGlvbi4gVGhlbiByZXBlYXQgd2FybmluZworCQkg KiB3aXRoIGV4cG9uZW50aWFsbHkgaW5jcmVhc2luZyBpbnRlcnZhbHMgdW50aWwgd2UgaXRl cmF0ZSBtb3JlIHRoYW4KKwkJICogR0VUTkVXT0lEX1dBUk5fTUFYX0lOVEVSVkFMLiBGaW5h bGx5IHJlcGVhdCB3YXJuaW5nIGV2ZXJ5CisJCSAqIEdFVE5FV09JRF9XQVJOX01BWF9JTlRF UlZBTCB1bmxlc3MgYW4gdW51c2VkIE9JRCBpcyBmb3VuZC4gVGhpcworCQkgKiBsb2dpYyBp cyBuZWNlc3Nhcnkgbm90IHRvIGZpbGwgdXAgdGhlIHNlcnZlciBsb2cgd2l0aCB0aGUgc2lt aWxhcgorCQkgKiBtZXNzYWdlcy4KKwkJICovCisJCWlmIChyZXRyaWVzID49IHJldHJpZXNf YmVmb3JlX3dhcm4pCisJCXsKKwkJCWVyZXBvcnQoV0FSTklORywKKwkJCQkJKGVycm1zZygi c3RpbGwgZmluZGluZyBhbiB1bnVzZWQgT0lEIHdpdGhpbiByZWxhdGlvbiBcIiVzXCIiLAor CQkJCQkJCVJlbGF0aW9uR2V0UmVsYXRpb25OYW1lKHJlbGF0aW9uKSksCisJCQkJCSBlcnJk ZXRhaWwoIk9JRCBjYW5kaWRhdGVzIHdlcmUgY2hlY2tlZCBcIiVsbHVcIiAgdGltZXMsIGJ1 dCBubyB1bnVzZWQgT0lEIGlzIHlldCBmb3VuZC4iLAorCQkJCQkJCSAgICh1bnNpZ25lZCBs b25nIGxvbmcpIHJldHJpZXMpKSk7CisKKwkJCS8qCisJCQkgKiBEb3VibGUgdGhlIG51bWJl ciBvZiByZXRyaWVzIHRvIGRvIGJlZm9yZSB3YXJuaW5nIG5leHQgdW50aWwgaXQKKwkJCSAq IHJlYWNoZXMgR0VUTkVXT0lEX1dBUk5fTUFYX0lOVEVSVkFMLgorCQkJICovCisJCQlpZiAo cmV0cmllc19iZWZvcmVfd2FybiAqIDIgPD0gR0VUTkVXT0lEX1dBUk5fTUFYX0lOVEVSVkFM KQorCQkJCXJldHJpZXNfYmVmb3JlX3dhcm4gKj0gMjsKKwkJCWVsc2UKKwkJCQlyZXRyaWVz X2JlZm9yZV93YXJuICs9IEdFVE5FV09JRF9XQVJOX01BWF9JTlRFUlZBTDsKKwkJfQorCisJ CXJldHJpZXMrKzsKIAl9IHdoaWxlIChjb2xsaWRlcyk7CiAKKwkvKgorCSAqIElmIGF0IGxl YXN0IG9uZSB3YXJuaW5nIGlzIGVtaXR0ZWQsIGFsc28gd2FybiB0aGUgY29tcGxldGlvbiBv ZiBPSUQKKwkgKiBhc3NpZ25tZW50LgorCSAqLworCWlmIChyZXRyaWVzID4gR0VUTkVXT0lE X1dBUk5fVEhSRVNIT0xEKQorCXsKKwkJZXJlcG9ydChXQVJOSU5HLAorCQkJCShlcnJtc2co Im5ldyBPSUQgaGFzIGJlZW4gYXNzaWduZWQgaW4gcmVsYXRpb24gXCIlc1wiIGFmdGVyIFwi JWxsdVwiIHJldHJpZXMiLAorCQkJCQkJUmVsYXRpb25HZXRSZWxhdGlvbk5hbWUocmVsYXRp b24pLCAodW5zaWduZWQgbG9uZyBsb25nKSByZXRyaWVzKSkpOworCX0KKwogCXJldHVybiBu ZXdPaWQ7CiB9CiAK --------------54926145FE0A55F768D1D710--