Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1q01di-0001W8-Po for pgsql-hackers@arkaria.postgresql.org; Fri, 19 May 2023 15:03:30 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1q01dh-0007Bb-La for pgsql-hackers@arkaria.postgresql.org; Fri, 19 May 2023 15:03:29 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1q01dh-0007BS-7Y for pgsql-hackers@lists.postgresql.org; Fri, 19 May 2023 15:03:29 +0000 Received: from new3-smtp.messagingengine.com ([66.111.4.229]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1q01dd-000gug-Ic for pgsql-hackers@postgresql.org; Fri, 19 May 2023 15:03:28 +0000 Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailnew.nyi.internal (Postfix) with ESMTP id 7A05A580ADB; Fri, 19 May 2023 11:03:24 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Fri, 19 May 2023 11:03:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1684508604; x=1684515804; bh=hz0hnQc68Bb1C +l7RFlE/TDzt1H8/bJD3ZPawSbo9A0=; b=E1/qds9qDSRwCf5lc25yh7FfnKfrY 3pos4G1LekB0FYRYby3ap5owPkHk9t9A9uoxFYVbZVoNO+FNdQHKqkIV7Lz//hOe 5Z41ynwQcQ4l5L72kKv51tCe/3cXuiv+7LphIuCuYyDqOIBM+g+XLIulo2Oarl9F FTvs7T2NURc95rFUA0TZ8wAqcgFj+3A7TYlC7nKSJxwueRxM+iQiW+XpxAtPQsUJ lo6E7vIF4CbhZPcvymJXv3s1RktwO/QMwTEUHB+MqUsj3bs9UCoPzumTTlwvFVew lnCqk1QxfIKnXSv9AELSZ1DDiZGua05E1hWL3NBl6IIYKcTEr4nkjlZ5Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfeeihedgkeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptgfkffggfgfuvfevfhfhjgesmhdtreertdefjeenucfhrhhomheprfgvthgv rhcugfhishgvnhhtrhgruhhtuceophgvthgvrhdrvghishgvnhhtrhgruhhtsegvnhhtvg hrphhrihhsvggusgdrtghomheqnecuggftrfgrthhtvghrnhepuedtkeehueehtddvgfff heelteefhfeifeduiedtffeitdeludejhfejudefffffnecuvehluhhsthgvrhfuihiivg eptdenucfrrghrrghmpehmrghilhhfrhhomhepphgvthgvrhdrvghishgvnhhtrhgruhht segvnhhtvghrphhrihhsvggusgdrtghomh X-ME-Proxy: Feedback-ID: i131946ab:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 19 May 2023 11:03:23 -0400 (EDT) Content-Type: multipart/mixed; boundary="------------7XClIbdPjpK4f3KkFsXvJYYx" Message-ID: Date: Fri, 19 May 2023 17:03:21 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.1 Subject: Re: smgrzeroextend clarification Content-Language: en-US To: Andres Freund Cc: pgsql-hackers References: <22fed8ba-01c3-2008-a256-4ea912d68fab@enterprisedb.com> <20230510181049.ad5vw6kijqd2y274@awork3.anarazel.de> <20230516233808.m2rzhsudg7cwh6fx@awork3.anarazel.de> From: Peter Eisentraut In-Reply-To: <20230516233808.m2rzhsudg7cwh6fx@awork3.anarazel.de> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------7XClIbdPjpK4f3KkFsXvJYYx Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit I have committed some of the unrelated formatting changes separately, so what's left now is attached. On 17.05.23 01:38, Andres Freund wrote: >> - I left this for smgrzeroextend(): >> >> FIXME: why both blocknum and nblocks > > I guess you're suggesting that we would do an lstat() to figure out the size > instead? Or use some cached size? That'd not be trivial to add - and it just > seems unrelated to smgzerorextend(), it's just as true for smgrextend(). What smgzerorextend() does now seems sensible to me. I'd just like one or two sentences that document its API. Right now, blocknum and nblocks are not documented at all. Of course, we can guess, but I'm also interested in edge cases. What are you supposed to pass for blocknum? What happens if it's not right after the current last block? You answered that, but is that just what happens to happen, or do we actually want to support that? What happens if blocknum is *before* the currently last block? Would that overwrite the last existing blocks with zero? What if nblocks is negative? Does that zero out blocks backwards? Obviously, the answer for most of these right now is that you're not supposed to do that. But as the smgrextend() + hash index case shows, these things tend to grow in unexpected directions. Also, slightly unrelated to the API, did we consider COW file systems? Like, is explicitly allocating blocks of zeroes sensible on btrfs? --------------7XClIbdPjpK4f3KkFsXvJYYx Content-Type: text/plain; charset=UTF-8; name="v2-0001-WIP-Improve-smgr-source-code-comments.patch" Content-Disposition: attachment; filename="v2-0001-WIP-Improve-smgr-source-code-comments.patch" Content-Transfer-Encoding: base64 RnJvbSAxN2ZmYzYwNTVjNzc1NWQxMTBjYTE5ZjIxYmMzMjhiZjE5MTk3ODk2IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQZXRlciBFaXNlbnRyYXV0IDxwZXRlckBlaXNlbnRy YXV0Lm9yZz4KRGF0ZTogRnJpLCAxOSBNYXkgMjAyMyAxNjo0OTowMyArMDIwMApTdWJqZWN0 OiBbUEFUQ0ggdjJdIFdJUDogSW1wcm92ZSBzbWdyIHNvdXJjZSBjb2RlIGNvbW1lbnRzCgpE aXNjdXNzaW9uOiBodHRwczovL3d3dy5wb3N0Z3Jlc3FsLm9yZy9tZXNzYWdlLWlkL2ZsYXQv MjJmZWQ4YmEtMDFjMy0yMDA4LWEyNTYtNGVhOTEyZDY4ZmFiJTQwZW50ZXJwcmlzZWRiLmNv bQotLS0KIHNyYy9iYWNrZW5kL3N0b3JhZ2Uvc21nci9tZC5jICAgfCAxNyArKysrKy0tLS0t LS0tLS0tLQogc3JjL2JhY2tlbmQvc3RvcmFnZS9zbWdyL3NtZ3IuYyB8IDI0ICsrKysrKysr KysrKysrKystLS0tLS0tLQogMiBmaWxlcyBjaGFuZ2VkLCAyMSBpbnNlcnRpb25zKCspLCAy MCBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9zcmMvYmFja2VuZC9zdG9yYWdlL3NtZ3Iv bWQuYyBiL3NyYy9iYWNrZW5kL3N0b3JhZ2Uvc21nci9tZC5jCmluZGV4IDQyZTM1MDEyNTUu LmExOWJhYjQ3ZjIgMTAwNjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL3N0b3JhZ2Uvc21nci9tZC5j CisrKyBiL3NyYy9iYWNrZW5kL3N0b3JhZ2Uvc21nci9tZC5jCkBAIC00NDksMTEgKzQ0OSwx MSBAQCBtZHVubGlua2ZvcmsoUmVsRmlsZUxvY2F0b3JCYWNrZW5kIHJsb2NhdG9yLCBGb3Jr TnVtYmVyIGZvcmtudW0sIGJvb2wgaXNSZWRvKQogLyoKICAqIG1kZXh0ZW5kKCkgLS0gQWRk IGEgYmxvY2sgdG8gdGhlIHNwZWNpZmllZCByZWxhdGlvbi4KICAqCi0gKiBUaGUgc2VtYW50 aWNzIGFyZSBuZWFybHkgdGhlIHNhbWUgYXMgbWR3cml0ZSgpOiB3cml0ZSBhdCB0aGUKLSAq IHNwZWNpZmllZCBwb3NpdGlvbi4gIEhvd2V2ZXIsIHRoaXMgaXMgdG8gYmUgdXNlZCBmb3Ig dGhlIGNhc2Ugb2YKLSAqIGV4dGVuZGluZyBhIHJlbGF0aW9uIChpLmUuLCBibG9ja251bSBp cyBhdCBvciBiZXlvbmQgdGhlIGN1cnJlbnQKLSAqIEVPRikuICBOb3RlIHRoYXQgd2UgYXNz dW1lIHdyaXRpbmcgYSBibG9jayBiZXlvbmQgY3VycmVudCBFT0YKLSAqIGNhdXNlcyBpbnRl cnZlbmluZyBmaWxlIHNwYWNlIHRvIGJlY29tZSBmaWxsZWQgd2l0aCB6ZXJvZXMuCisgKiBU aGlzIGlzIHRvIGJlIHVzZWQgZm9yIHRoZSBjYXNlIG9mIGV4dGVuZGluZyBhIHJlbGF0aW9u IChpLmUuLCBibG9ja251bSBpcworICogYXQgb3IgYmV5b25kIHRoZSBjdXJyZW50IEVPRiku ICBOb3RlIHRoYXQgd3JpdGluZyBhIGJsb2NrIGJleW9uZCBjdXJyZW50CisgKiBFT0YgbXVz dCBjYXVzZSB0aGUgaW50ZXJ2ZW5pbmcgZmlsZSBzcGFjZSB0byBiZWNvbWUgZmlsbGVkIHdp dGggemVyb2VzLgorICogVGhlIFBPU0lYIGZpbGUgc3lzdGVtIEFQSXMgZG8gdGhhdCBhdXRv bWF0aWNhbGx5LCBzbyB3ZSBkb24ndCBuZWVkIHRvIGRvCisgKiBhbnl0aGluZyBhYm91dCB0 aGF0LgogICovCiB2b2lkCiBtZGV4dGVuZChTTWdyUmVsYXRpb24gcmVsbiwgRm9ya051bWJl ciBmb3JrbnVtLCBCbG9ja051bWJlciBibG9ja251bSwKQEAgLTUxNiw5ICs1MTYsNiBAQCBt ZGV4dGVuZChTTWdyUmVsYXRpb24gcmVsbiwgRm9ya051bWJlciBmb3JrbnVtLCBCbG9ja051 bWJlciBibG9ja251bSwKIAogLyoKICAqIG1kemVyb2V4dGVuZCgpIC0tIEFkZCBuZXcgemVy b2VkIG91dCBibG9ja3MgdG8gdGhlIHNwZWNpZmllZCByZWxhdGlvbi4KLSAqCi0gKiBTaW1p bGFyIHRvIG1kZXh0ZW5kKCksIGV4Y2VwdCB0aGUgcmVsYXRpb24gY2FuIGJlIGV4dGVuZGVk IGJ5IG11bHRpcGxlCi0gKiBibG9ja3MgYXQgb25jZSBhbmQgdGhlIGFkZGVkIGJsb2NrcyB3 aWxsIGJlIGZpbGxlZCB3aXRoIHplcm9lcy4KICAqLwogdm9pZAogbWR6ZXJvZXh0ZW5kKFNN Z3JSZWxhdGlvbiByZWxuLCBGb3JrTnVtYmVyIGZvcmtudW0sCkBAIC04MDAsMTAgKzc5Nyw2 IEBAIG1kcmVhZChTTWdyUmVsYXRpb24gcmVsbiwgRm9ya051bWJlciBmb3JrbnVtLCBCbG9j a051bWJlciBibG9ja251bSwKIAogLyoKICAqIG1kd3JpdGUoKSAtLSBXcml0ZSB0aGUgc3Vw cGxpZWQgYmxvY2sgYXQgdGhlIGFwcHJvcHJpYXRlIGxvY2F0aW9uLgotICoKLSAqIFRoaXMg aXMgdG8gYmUgdXNlZCBvbmx5IGZvciB1cGRhdGluZyBhbHJlYWR5LWV4aXN0aW5nIGJsb2Nr cyBvZiBhCi0gKiByZWxhdGlvbiAoaWUsIHRob3NlIGJlZm9yZSB0aGUgY3VycmVudCBFT0Yp LiAgVG8gZXh0ZW5kIGEgcmVsYXRpb24sCi0gKiB1c2UgbWRleHRlbmQoKS4KICAqLwogdm9p ZAogbWR3cml0ZShTTWdyUmVsYXRpb24gcmVsbiwgRm9ya051bWJlciBmb3JrbnVtLCBCbG9j a051bWJlciBibG9ja251bSwKZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3N0b3JhZ2Uvc21n ci9zbWdyLmMgYi9zcmMvYmFja2VuZC9zdG9yYWdlL3NtZ3Ivc21nci5jCmluZGV4IGY3NmM0 NjA1ZGIuLmM0YzZjNWEzNzMgMTAwNjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL3N0b3JhZ2Uvc21n ci9zbWdyLmMKKysrIGIvc3JjL2JhY2tlbmQvc3RvcmFnZS9zbWdyL3NtZ3IuYwpAQCAtMzAs NyArMzAsOSBAQAogCiAvKgogICogVGhpcyBzdHJ1Y3Qgb2YgZnVuY3Rpb24gcG9pbnRlcnMg ZGVmaW5lcyB0aGUgQVBJIGJldHdlZW4gc21nci5jIGFuZAotICogYW55IGluZGl2aWR1YWwg c3RvcmFnZSBtYW5hZ2VyIG1vZHVsZS4gIE5vdGUgdGhhdCBzbWdyIHN1YmZ1bmN0aW9ucyBh cmUKKyAqIGFueSBpbmRpdmlkdWFsIHN0b3JhZ2UgbWFuYWdlciBtb2R1bGUuCisgKgorICog Tm90ZSB0aGF0IHNtZ3Igc3ViZnVuY3Rpb25zIGFyZQogICogZ2VuZXJhbGx5IGV4cGVjdGVk IHRvIHJlcG9ydCBwcm9ibGVtcyB2aWEgZWxvZyhFUlJPUikuICBBbiBleGNlcHRpb24gaXMK ICAqIHRoYXQgc21ncl91bmxpbmsgc2hvdWxkIHVzZSBlbG9nKFdBUk5JTkcpLCByYXRoZXIg dGhhbiBlcnJvcmluZyBvdXQsCiAgKiBiZWNhdXNlIHdlIG5vcm1hbGx5IHVubGluayByZWxh dGlvbnMgZHVyaW5nIHBvc3QtY29tbWl0L2Fib3J0IGNsZWFudXAsCkBAIC0xMDQsOCArMTA2 LDcgQEAgc3RhdGljIHZvaWQgc21ncnNodXRkb3duKGludCBjb2RlLCBEYXR1bSBhcmcpOwog CiAKIC8qCi0gKiBzbWdyaW5pdCgpLCBzbWdyc2h1dGRvd24oKSAtLSBJbml0aWFsaXplIG9y IHNodXQgZG93biBzdG9yYWdlCi0gKgkJCQkJCQkJIG1hbmFnZXJzLgorICogc21ncmluaXQo KSAtLSBJbml0aWFsaXplIHN0b3JhZ2UgbWFuYWdlcnMuCiAgKgogICogTm90ZTogc21ncmlu aXQgaXMgY2FsbGVkIGR1cmluZyBiYWNrZW5kIHN0YXJ0dXAgKG5vcm1hbCBvciBzdGFuZGFs b25lCiAgKiBjYXNlKSwgKm5vdCogZHVyaW5nIHBvc3RtYXN0ZXIgc3RhcnQuICBUaGVyZWZv cmUsIGFueSByZXNvdXJjZXMgY3JlYXRlZApAQCAtMTQ0LDYgKzE0NSw4IEBAIHNtZ3JzaHV0 ZG93bihpbnQgY29kZSwgRGF0dW0gYXJnKQogLyoKICAqIHNtZ3JvcGVuKCkgLS0gUmV0dXJu IGFuIFNNZ3JSZWxhdGlvbiBvYmplY3QsIGNyZWF0aW5nIGl0IGlmIG5lZWQgYmUuCiAgKgor ICogImJhY2tlbmQiIGlzIGZvciB0ZW1wb3JhcnkgdGFibGVzLCBvdGhlcndpc2UgSW52YWxp ZEJhY2tlbmRJZC4KKyAqCiAgKiBUaGlzIGRvZXMgbm90IGF0dGVtcHQgdG8gYWN0dWFsbHkg b3BlbiB0aGUgdW5kZXJseWluZyBmaWxlLgogICovCiBTTWdyUmVsYXRpb24KQEAgLTM2OCw2 ICszNzEsOSBAQCBzbWdyY2xvc2VyZWxsb2NhdG9yKFJlbEZpbGVMb2NhdG9yQmFja2VuZCBy bG9jYXRvcikKICAqIEdpdmVuIGFuIGFscmVhZHktY3JlYXRlZCAoYnV0IHByZXN1bWFibHkg dW51c2VkKSBTTWdyUmVsYXRpb24sCiAgKiBjYXVzZSB0aGUgdW5kZXJseWluZyBkaXNrIGZp bGUgb3Igb3RoZXIgc3RvcmFnZSBmb3IgdGhlIGZvcmsKICAqIHRvIGJlIGNyZWF0ZWQuCisg KgorICogaXNSZWRvIGlzIHRydWUgZHVyaW5nIHJlY292ZXJ5LiAgSW4gdGhhdCBjYXNlLCB0 aGUgdW5kZXJseWluZyBzdG9yYWdlIG1heQorICogYWxyZWFkeSBleGlzdC4KICAqLwogdm9p ZAogc21ncmNyZWF0ZShTTWdyUmVsYXRpb24gcmVsbiwgRm9ya051bWJlciBmb3JrbnVtLCBi b29sIGlzUmVkbykKQEAgLTQ5MCw4ICs0OTYsOCBAQCBzbWdyZG91bmxpbmthbGwoU01nclJl bGF0aW9uICpyZWxzLCBpbnQgbnJlbHMsIGJvb2wgaXNSZWRvKQogICogVGhlIHNlbWFudGlj cyBhcmUgbmVhcmx5IHRoZSBzYW1lIGFzIHNtZ3J3cml0ZSgpOiB3cml0ZSBhdCB0aGUKICAq IHNwZWNpZmllZCBwb3NpdGlvbi4gIEhvd2V2ZXIsIHRoaXMgaXMgdG8gYmUgdXNlZCBmb3Ig dGhlIGNhc2Ugb2YKICAqIGV4dGVuZGluZyBhIHJlbGF0aW9uIChpLmUuLCBibG9ja251bSBp cyBhdCBvciBiZXlvbmQgdGhlIGN1cnJlbnQKLSAqIEVPRikuICBOb3RlIHRoYXQgd2UgYXNz dW1lIHdyaXRpbmcgYSBibG9jayBiZXlvbmQgY3VycmVudCBFT0YKLSAqIGNhdXNlcyBpbnRl cnZlbmluZyBmaWxlIHNwYWNlIHRvIGJlY29tZSBmaWxsZWQgd2l0aCB6ZXJvZXMuCisgKiBF T0YpLiAgV3JpdGluZyBhIGJsb2NrIGJleW9uZCBjdXJyZW50IEVPRiBtdXN0IGNhdXNlIHRo ZQorICogaW50ZXJ2ZW5pbmcgZmlsZSBzcGFjZSB0byBiZWNvbWUgZmlsbGVkIHdpdGggemVy b2VzLgogICovCiB2b2lkCiBzbWdyZXh0ZW5kKFNNZ3JSZWxhdGlvbiByZWxuLCBGb3JrTnVt YmVyIGZvcmtudW0sIEJsb2NrTnVtYmVyIGJsb2NrbnVtLApAQCAtNTE0LDkgKzUyMCwxMSBA QCBzbWdyZXh0ZW5kKFNNZ3JSZWxhdGlvbiByZWxuLCBGb3JrTnVtYmVyIGZvcmtudW0sIEJs b2NrTnVtYmVyIGJsb2NrbnVtLAogLyoKICAqIHNtZ3J6ZXJvZXh0ZW5kKCkgLS0gQWRkIG5l dyB6ZXJvZWQgb3V0IGJsb2NrcyB0byBhIGZpbGUuCiAgKgotICogU2ltaWxhciB0byBzbWdy ZXh0ZW5kKCksIGV4Y2VwdCB0aGUgcmVsYXRpb24gY2FuIGJlIGV4dGVuZGVkIGJ5Ci0gKiBt dWx0aXBsZSBibG9ja3MgYXQgb25jZSBhbmQgdGhlIGFkZGVkIGJsb2NrcyB3aWxsIGJlIGZp bGxlZCB3aXRoCi0gKiB6ZXJvZXMuCisgKiBFeHRlbmQgdGhlIHJlbGF0aW9uIGJ5IHRoZSBn aXZlbiBudW1iZXIgb2YgYmxvY2tzLCB3aGljaCB3aWxsIGJlIGZpbGxlZAorICogd2l0aCB6 ZXJvZXMuICBUaGlzIGlzIHNpbWlsYXIgdG8gc21ncmV4dGVuZCgpIGJ1dCBvbmx5IGV4dGVu ZHMgYW5kIGRvZXMgbm90CisgKiB3cml0ZSBhIGJ1ZmZlciBvZiBkYXRhLgorICoKKyAqIEZJ WE1FOiB3aHkgYm90aCBibG9ja251bSBhbmQgbmJsb2NrcwogICovCiB2b2lkCiBzbWdyemVy b2V4dGVuZChTTWdyUmVsYXRpb24gcmVsbiwgRm9ya051bWJlciBmb3JrbnVtLCBCbG9ja051 bWJlciBibG9ja251bSwKLS0gCjIuNDAuMQoK --------------7XClIbdPjpK4f3KkFsXvJYYx--