Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rrOYI-000LXd-HU for pgsql-committers@arkaria.postgresql.org; Mon, 01 Apr 2024 20:46:47 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rrOYG-001Zh9-RZ for pgsql-committers@arkaria.postgresql.org; Mon, 01 Apr 2024 20:46:45 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rrOYG-001Zh1-Fx for pgsql-committers@lists.postgresql.org; Mon, 01 Apr 2024 20:46:45 +0000 Received: from mahout.postgresql.org ([2001:4800:3e1:1::227]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rrOYE-0001oU-43 for pgsql-committers@lists.postgresql.org; Mon, 01 Apr 2024 20:46:43 +0000 Received: from gemulon.postgresql.org ([2001:4800:3e1:1::198]) by mahout.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rrOYD-0004g2-3a for pgsql-committers@lists.postgresql.org; Mon, 01 Apr 2024 20:46:41 +0000 Received: from localhost ([127.0.0.1] helo=gemulon.postgresql.org) by gemulon.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rrOYC-00020M-3q for pgsql-committers@lists.postgresql.org; Mon, 01 Apr 2024 20:46:40 +0000 Content-Type: multipart/mixed; boundary="===============0915861725372290970==" MIME-Version: 1.0 From: Tom Lane To: pgsql-committers@lists.postgresql.org Subject: pgsql: Rearrange pg_dump's handling of large objects for better efficie X-Auto-Response-Suppress: All Auto-Submitted: auto-generated Message-Id: Date: Mon, 01 Apr 2024 20:46:40 +0000 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --===============0915861725372290970== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 UmVhcnJhbmdlIHBnX2R1bXAncyBoYW5kbGluZyBvZiBsYXJnZSBvYmplY3RzIGZvciBiZXR0ZXIg ZWZmaWNpZW5jeS4KCkNvbW1pdCBjMGQ1YmU1ZDYgY2F1c2VkIHBnX2R1bXAgdG8gY3JlYXRlIGEg c2VwYXJhdGUgQkxPQiBtZXRhZGF0YSBUT0MKZW50cnkgZm9yIGVhY2ggbGFyZ2Ugb2JqZWN0IChi bG9iKSwgYnV0IGl0IGRpZCBub3QgdG91Y2ggdGhlIGFuY2llbnQKZGVjaXNpb24gdG8gcHV0IGFs bCB0aGUgYmxvYnMnIGRhdGEgaW50byBhIHNpbmdsZSAiQkxPQlMiIFRPQyBlbnRyeS4KVGhpcyBp cyBiYWQgZm9yIGEgZmV3IHJlYXNvbnM6IGZvciBkYXRhYmFzZXMgd2l0aCBtaWxsaW9ucyBvZiBi bG9icywKdGhlIFRPQyBiZWNvbWVzIHVucmVhc29uYWJseSBsYXJnZSwgY2F1c2luZyBwZXJmb3Jt YW5jZSBpc3N1ZXM7CnNlbGVjdGl2ZSByZXN0b3JlIG9mIGp1c3Qgc29tZSBibG9icyBpcyBxdWl0 ZSBpbXBvc3NpYmxlOyBhbmQgd2UKY2Fubm90IHBhcmFsbGVsaXplIGVpdGhlciBkdW1wIG9yIHJl c3RvcmUgb2YgdGhlIGJsb2IgZGF0YSwgc2luY2Ugb3VyCmFyY2hpdGVjdHVyZSBmb3IgdGhhdCBy ZWxpZXMgb24gZmFybWluZyBvdXQgd2hvbGUgVE9DIGVudHJpZXMgdG8Kd29ya2VyIHByb2Nlc3Nl cy4KClRvIGltcHJvdmUgbWF0dGVycywgbGV0J3MgZ3JvdXAgbXVsdGlwbGUgYmxvYnMgaW50byBl YWNoIGJsb2IgbWV0YWRhdGEKVE9DIGVudHJ5LCBhbmQgdGhlbiBtYWtlIGNvcnJlc3BvbmRpbmcg cGVyLWdyb3VwIGJsb2IgZGF0YSBUT0MgZW50cmllcy4KU2VsZWN0aXZlIHJlc3RvcmUgdXNpbmcg cGdfcmVzdG9yZSdzIC1sLy1MIHN3aXRjaGVzIGlzIHRoZW4gcG9zc2libGUsCnRob3VnaCBvbmx5 IGF0IHRoZSBncm91cCBsZXZlbC4gIChQZXJoYXBzIHdlIHNob3VsZCBwcm92aWRlIGEgc3dpdGNo CnRvIGFsbG93IGZvcmNpbmcgb25lLWJsb2ItcGVyLWdyb3VwIGZvciB1c2VycyB3aG8gbmVlZCBw cmVjaXNlCnNlbGVjdGl2ZSByZXN0b3JlIGFuZCBkb24ndCBoYXZlIGh1Z2UgbnVtYmVycyBvZiBi bG9icy4gIFRoaXMgcGF0Y2gKZG9lc24ndCBkbyB0aGF0LCBpbnN0ZWFkIGp1c3QgaGFyZC13aXJp bmcgdGhlIG1heGltdW0gbnVtYmVyIG9mIGJsb2JzCnBlciBlbnRyeSBhdCAxMDAwLikKClRoZSBi bG9icyBpbiBhIGdyb3VwIG11c3QgYWxsIGhhdmUgdGhlIHNhbWUgb3duZXIsIHNpbmNlIHRoZSBU T0MgZW50cnkKZm9ybWF0IG9ubHkgYWxsb3dzIG9uZSBvd25lciB0byBiZSBuYW1lZC4gIEluIHRo aXMgaW1wbGVtZW50YXRpb24Kd2UgYWxzbyByZXF1aXJlIHRoZW0gdG8gYWxsIHNoYXJlIHRoZSBz YW1lIEFDTCAoZ3JhbnRzKTsgdGhlIGFyY2hpdmUKZm9ybWF0IHdvdWxkbid0IHJlcXVpcmUgdGhh dCwgYnV0IHBnX2R1bXAncyByZXByZXNlbnRhdGlvbiBvZgpEdW1wYWJsZU9iamVjdHMgZG9lcy4g IEl0IHNlZW1zIHVubGlrZWx5IHRoYXQgZWl0aGVyIHJlc3RyaWN0aW9uCndpbGwgYmUgcHJvYmxl bWF0aWMgZm9yIGRhdGFiYXNlcyB3aXRoIGh1Z2UgbnVtYmVycyBvZiBibG9icy4KClRoZSBtZXRh ZGF0YSBUT0MgZW50cmllcyBub3cgaGF2ZSBhICJkZXNjIiBzdHJpbmcgb2YgIkJMT0IgTUVUQURB VEEiLAphbmQgdGhlaXIgImRlZm4iIHN0cmluZyBpcyBqdXN0IGEgbmV3bGluZS1zZXBhcmF0ZWQg bGlzdCBvZiBibG9iIE9JRHMuClRoZSByZXN0b3JlIGNvZGUgaGFzIHRvIGdlbmVyYXRlIGNyZWF0 aW9uIGNvbW1hbmRzLCBBTFRFUiBPV05FUgpjb21tYW5kcywgYW5kIGRyb3AgY29tbWFuZHMgKGZv ciAtLWNsZWFuIG1vZGUpIGZyb20gdGhhdC4gIFdlIHdvdWxkCm5lZWQgc3BlY2lhbC1jYXNlIGNv ZGUgZm9yIEFMVEVSIE9XTkVSIGFuZCBkcm9wIGluIGFueSBjYXNlLCBzbyB0aGUKYWx0ZXJuYXRp dmUgb2Yga2VlcGluZyB0aGUgImRlZm4iIGFzIGRpcmVjdGx5IGV4ZWN1dGFibGUgU1FMIGNvZGUK Zm9yIGNyZWF0aW9uIHdvdWxkbid0IGJ1eSBtdWNoLCBhbmQgaXQgc2VlbXMgbGlrZSBpdCdkIGJs b2F0IHRoZQphcmNoaXZlIHRvIGxpdHRsZSBwdXJwb3NlLgoKU2luY2Ugd2UgcmVxdWlyZSB0aGUg YmxvYnMgb2YgYSBtZXRhZGF0YSBncm91cCB0byBzaGFyZSB0aGUgc2FtZSBBQ0wsCndlIGNhbiBm dXJ0aGVybW9yZSBzdG9yZSBvbmx5IG9uZSBjb3B5IG9mIHRoYXQgQUNMLCBhbmQgdGhlbiBtYWtl CnBnX3Jlc3RvcmUgcmVnZW5lcmF0ZSB0aGUgYXBwcm9wcmlhdGUgY29tbWFuZHMgZm9yIGVhY2gg YmxvYi4gIFRoaXMKc2F2ZXMgc3BhY2UgaW4gdGhlIGR1bXAgZmlsZSBub3Qgb25seSBieSByZW1v dmluZyBkdXBsaWNhdGl2ZSBTUUwKY29tbWFuZCBzdHJpbmdzLCBidXQgYnkgbm90IG5lZWRpbmcg YSBzZXBhcmF0ZSBUT0MgZW50cnkgZm9yIGVhY2gKYmxvYidzIEFDTC4gIEluIHR1cm4sIHRoYXQg cmVkdWNlcyBjbGllbnQtc2lkZSBtZW1vcnkgcmVxdWlyZW1lbnRzIGZvcgpoYW5kbGluZyBtYW55 IGJsb2JzLgoKQUNMIFRPQyBlbnRyaWVzIHRoYXQgbmVlZCB0aGlzIHNwZWNpYWwgcHJvY2Vzc2lu ZyBhcmUgbGFiZWxlZCBhcwoiQUNMIi8iTEFSR0UgT0JKRUNUUyBubm4uLm5ubiIuICBJZiB3ZSBo YXZlIGEgYmxvYiB3aXRoIGEgdW5pcXVlIEFDTCwKY29udGludWUgdG8gbGFiZWwgaXQgYXMgIkFD TCIvIkxBUkdFIE9CSkVDVCBubm4iLiAgV2UgZG9uJ3QgYWN0dWFsbHkKaGF2ZSB0byBtYWtlIHN1 Y2ggYSBkaXN0aW5jdGlvbiwgYnV0IGl0IHNhdmVzIGEgZmV3IGN5Y2xlcyBkdXJpbmcKcmVzdG9y ZSBmb3IgdGhlIGVhc3kgY2FzZSwgYW5kIGl0IHNlZW1zIGxpa2UgYSBnb29kIGlkZWEgdG8gbm90 IGNoYW5nZQp0aGUgVE9DIGNvbnRlbnRzIHVubmVjZXNzYXJpbHkuCgpUaGUgZGF0YSBUT0MgZW50 cmllcyAoIkJMT0JTIikgYXJlIGV4YWN0bHkgdGhlIHNhbWUgYXMgYmVmb3JlLApleGNlcHQgdGhh dCBub3cgdGhlcmUgY2FuIGJlIG1vcmUgdGhhbiBvbmUsIHNvIHdlJ2QgYmV0dGVyIGdpdmUgdGhl bQppZGVudGlmeWluZyB0YWcgc3RyaW5ncy4KCkFsc28sIGNvbW1pdCBjMGQ1YmU1ZDYgcHV0IHRo ZSBuZXcgQkxPQiBtZXRhZGF0YSBUT0MgZW50cmllcyBpbnRvClNFQ1RJT05fUFJFX0RBVEEsIHdo aWNoIHBlcmhhcHMgaXMgZGVmZW5zaWJsZSBpbiBzb21lIHdheXMsIGJ1dAppdCdzIGEgcmF0aGVy IG9kZCBjaG9pY2UgY29uc2lkZXJpbmcgdGhhdCB3ZSBnbyBvdXQgb2Ygb3VyIHdheSB0bwp0cmVh dCBibG9icyBhcyBkYXRhLiAgTW9yZW92ZXIsIGJlY2F1c2UgcGFyYWxsZWwgcmVzdG9yZSBoYW5k bGVzCnRoZSBQUkVfREFUQSBzZWN0aW9uIHNlcmlhbGx5LCB0aGlzIG1lYW5zIHdlJ2Qgb25seSBn ZXQgcGFydCBvZiB0aGUKcGFyYWxsZWxpc20gc3BlZWR1cCB3ZSBjb3VsZCBob3BlIGZvci4gIE1v dmUgdGhlc2UgZW50cmllcyBpbnRvClNFQ1RJT05fREFUQSwgbGV0dGluZyB1cyBwYXJhbGxlbGl6 ZSB0aGUgbG9fY3JlYXRlIGNhbGxzIG5vdCBqdXN0IHRoZQpkYXRhIGxvYWRpbmcgd2hlbiB0aGVy ZSBhcmUgbWFueSBibG9icy4gIEFkZCBkZXBlbmRlbmNpZXMgdG8gZW5zdXJlCnRoYXQgd2Ugd29u J3QgdHJ5IHRvIGxvYWQgZGF0YSBmb3IgYSBibG9iIHdlJ3ZlIG5vdCB5ZXQgY3JlYXRlZC4KCkFz IHRoaXMgc3RhbmRzLCB3ZSBzdGlsbCBnZW5lcmF0ZSBhIHNlcGFyYXRlIFRPQyBlbnRyeSBmb3Ig YW55IGNvbW1lbnQKb3Igc2VjdXJpdHkgbGFiZWwgYXR0YWNoZWQgdG8gYSBibG9iLiAgSSBmZWVs IGNvbWZvcnRhYmxlIGluIGJlbGlldmluZwp0aGF0IGNvbW1lbnRzIGFuZCBzZWN1cml0eSBsYWJl bHMgb24gYmxvYnMgYXJlIHJhcmUsIHNvIHRoaXMgcGF0Y2gKc2hvdWxkIGJlIGVub3VnaCB0byBn ZXQgbW9zdCBvZiB0aGUgdXNlZnVsIFRPQyBjb21wcmVzc2lvbiBmb3IgYmxvYnMuCgpXZSBoYXZl IHRvIGJ1bXAgdGhlIGFyY2hpdmUgZmlsZSBmb3JtYXQgdmVyc2lvbiBudW1iZXIsIHNpbmNlIGV4 aXN0aW5nCnZlcnNpb25zIG9mIHBnX3Jlc3RvcmUgd291bGRuJ3Qga25vdyB0aGV5IG5lZWQgdG8g ZG8gc29tZXRoaW5nIHNwZWNpYWwKZm9yIEJMT0IgTUVUQURBVEEsIHBsdXMgdGhleSBhcmVuJ3Qg Z29pbmcgdG8gd29yayBjb3JyZWN0bHkgd2l0aAptdWx0aXBsZSBCTE9CUyBlbnRyaWVzIG9yIG11 bHRpcGxlLWxhcmdlLW9iamVjdCBBQ0wgZW50cmllcy4KClRoZSBkaXJlY3RvcnkgYW5kIHRhci1m aWxlIGZvcm1hdCBoYW5kbGVycyBuZWVkIHNvbWUgd29yawpmb3IgbXVsdGlwbGUgQkxPQlMgZW50 cmllczogdGhleSB1c2VkIHRvIGhhcmQtd2lyZSB0aGUgZmlsZSBuYW1lCmFzICJibG9icy50b2Mi LCB3aGljaCBpcyByZXBsYWNlZCBoZXJlIHdpdGggImJsb2JzXzxkdW1waWQ+LnRvYyIuClRoZSAw MDJfcGdfZHVtcC5wbCB0ZXN0IHNjcmlwdCBhbHNvIGtub3dzIGFib3V0IHRoYXQgYW5kIHJlcXVp cmVzCm1pbm9yIHVwZGF0ZXMuICAoSSBoYWQgdG8gZHJvcCB0aGUgdGVzdCBmb3IgbWFudWFsbHkt Y29tcHJlc3NlZApibG9icy50b2MgZmlsZXMgd2l0aCBMWjQsIGJlY2F1c2UgbHo0J3Mgb2J0dXNl IGNvbW1hbmQgbGluZQpkZXNpZ24gcmVxdWlyZXMgZXhwbGljaXQgc3BlY2lmaWNhdGlvbiBvZiB0 aGUgb3V0cHV0IGZpbGUgbmFtZQp3aGljaCBzZWVtcyBpbXByYWN0aWNhbCBoZXJlLiAgSSBkb24n dCB0aGluayB3ZSdyZSBsb3NpbmcgYW55CnVzZWZ1bCB0ZXN0IGNvdmVyYWdlIHRoZXJlYnk7IHRo YXQgdGVzdCBzdGFuemEgc2VlbXMgY29tcGxldGVseQpkdXBsaWNhdGl2ZSB3aXRoIHRoZSBnemlw IGFuZCB6c3RkIGNhc2VzIGFueXdheS4pCgpJbiBwYXNzaW5nLCBjZW50cmFsaXplIG1hbmFnZW1l bnQgb2YgdGhlIGxvX2J1ZiB1c2VkIHRvIGhvbGQgZGF0YQp3aGlsZSByZXN0b3JpbmcgYmxvYnMu ICBUaGUgY29kZSBwcmV2aW91c2x5IGhhZCBlYWNoIGZvcm1hdCBoYW5kbGVyCmNyZWF0ZSBsb19i dWYsIHdoaWNoIHNlZW1zIHJhdGhlciBwb2ludGxlc3MgZ2l2ZW4gdGhhdCB0aGUgZm9ybWF0Cmhh bmRsZXJzIGFsbCBtYWtlIGl0IHRoZSBzYW1lIHdheS4gIE1vcmVvdmVyLCB0aGUgZm9ybWF0IGhh bmRsZXJzCm5ldmVyIHVzZSBsb19idWYgZGlyZWN0bHksIG1ha2luZyB0aGlzIHNldHVwIGEgZmFp bHVyZSBmcm9tIGEKc2VwYXJhdGlvbi1vZi1jb25jZXJucyBzdGFuZHBvaW50LiAgTGV0J3MgbW92 ZSB0aGUgcmVzcG9uc2liaWxpdHkgaW50bwpwZ19iYWNrdXBfYXJjaGl2ZXIuYywgd2hpY2ggaXMg dGhlIG9ubHkgbW9kdWxlIGNvbmNlcm5lZCB3aXRoIGxvX2J1Zi4KVGhlIHJlYXNvbiB0byBkbyB0 aGlzIGluIHRoaXMgcGF0Y2ggaXMgdGhhdCBpdCBhbGxvd3MgYSBjZW50cmFsaXplZApmaXggZm9y IHRoZSBub3ctZmFsc2UgYXNzdW1wdGlvbiB0aGF0IHdlIG5ldmVyIHJlc3RvcmUgYmxvYnMgaW4K cGFyYWxsZWwuICBBbHNvLCBnZXQgcmlkIG9mIGRlYWQgY29kZSBpbiBEcm9wTE9JZkV4aXN0czog aXQncyBiZWVuIGEKbG9uZyB0aW1lIHNpbmNlIHdlIGhhZCBhbnkgbmVlZCB0byBiZSBhYmxlIHRv IHJlc3RvcmUgdG8gYSBwcmUtOS4wCnNlcnZlci4KCkRpc2N1c3Npb246IGh0dHBzOi8vcG9zdGdy LmVzL20vYTlmOTM3NmYxYzMzNDNhNmJiMzE5ZGNlMjk0ZTIwYWNARVgxM0QwNVVXQzAwMS5hbnQu YW1hem9uLmNvbQoKQnJhbmNoCi0tLS0tLQptYXN0ZXIKCkRldGFpbHMKLS0tLS0tLQpodHRwczov L2dpdC5wb3N0Z3Jlc3FsLm9yZy9wZy9jb21taXRkaWZmL2E0NWM3OGUzMjg0YjI2OTA4NWU5YTBj YmQwZWEzYjIzNmI3MTgwZmEKCk1vZGlmaWVkIEZpbGVzCi0tLS0tLS0tLS0tLS0tCnNyYy9iaW4v cGdfZHVtcC9jb21tb24uYyAgICAgICAgICAgICAgfCAgMjYgKysrCnNyYy9iaW4vcGdfZHVtcC9w Z19iYWNrdXBfYXJjaGl2ZXIuYyAgfCAxMDYgKysrKysrKy0tLQpzcmMvYmluL3BnX2R1bXAvcGdf YmFja3VwX2FyY2hpdmVyLmggIHwgICA3ICstCnNyYy9iaW4vcGdfZHVtcC9wZ19iYWNrdXBfY3Vz dG9tLmMgICAgfCAgMTEgKy0Kc3JjL2Jpbi9wZ19kdW1wL3BnX2JhY2t1cF9kYi5jICAgICAgICB8 IDEzMSArKysrKysrKysrLS0Kc3JjL2Jpbi9wZ19kdW1wL3BnX2JhY2t1cF9kaXJlY3RvcnkuYyB8 ICA0NCArKy0tCnNyYy9iaW4vcGdfZHVtcC9wZ19iYWNrdXBfbnVsbC5jICAgICAgfCAgIDggKy0K c3JjL2Jpbi9wZ19kdW1wL3BnX2JhY2t1cF90YXIuYyAgICAgICB8ICA0MyArKy0tCnNyYy9iaW4v cGdfZHVtcC9wZ19kdW1wLmMgICAgICAgICAgICAgfCAzNjYgKysrKysrKysrKysrKysrKysrKyst LS0tLS0tLS0tLS0tLQpzcmMvYmluL3BnX2R1bXAvcGdfZHVtcC5oICAgICAgICAgICAgIHwgIDEx ICsKc3JjL2Jpbi9wZ19kdW1wL3QvMDAyX3BnX2R1bXAucGwgICAgICB8ICAzOCArKy0tCjExIGZp bGVzIGNoYW5nZWQsIDUzMCBpbnNlcnRpb25zKCspLCAyNjEgZGVsZXRpb25zKC0pCgo= --===============0915861725372290970==--