Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1uP7Wn-000mXn-8w for pgsql-hackers@arkaria.postgresql.org; Tue, 10 Jun 2025 22:33:09 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1uP7Wl-00AnwZ-77 for pgsql-hackers@arkaria.postgresql.org; Tue, 10 Jun 2025 22:33:07 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1uP7Wk-00AnwP-TZ for pgsql-hackers@lists.postgresql.org; Tue, 10 Jun 2025 22:33:07 +0000 Received: from mout.gmx.net ([212.227.17.21]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1uP7Wj-001KE6-0N for pgsql-hackers@lists.postgresql.org; Tue, 10 Jun 2025 22:33:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmx.net; s=s31663417; t=1749594780; x=1750199580; i=jimis@gmx.net; bh=L0NeMglNfpWWi2S7jml/GOlYa5qM/YAmHVaj2p2aknk=; h=X-UI-Sender-Class:Date:From:To:cc:Subject:In-Reply-To:Message-ID: References:MIME-Version:Content-Type:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=OfndhJ8cu66qdum462srCj2gvoklNqRnENLEoLLPP2fIkk+js99ZKj3ayysTJq4B LwvCIy2eiDUtVxy8LdSKkyBs6TZ2X/HekgB23LCNkjlNozZXTX6PWyrJ7Opaxj+Fg R9nRV/ujAAWmv0HLR6wikyn+OVhYax18jxv5cVQMhN99ZEvQBfWkZ9zFZy78Ow5iK 8GSavk0LtJtJtfF3wlE9TvWFAO/fajJNtqWPwsnBWWQdZBgb6QZR0o6x3vUysBOu5 uETaAPY131FE0WKRK5YGxt51TCfAPhN6Sg79iGjieDbvEi+gXlAMHFsPKTGjHpKA2 tiWJHYDJH0/62K8Hyw== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from qtjimis-OpenSUSE.lan ([84.215.109.181]) by mail.gmx.net (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MVeI8-1uHSZz3gv6-00Ho0G; Wed, 11 Jun 2025 00:32:59 +0200 Date: Wed, 11 Jun 2025 00:32:58 +0200 (CEST) From: Dimitrios Apostolou To: Nathan Bossart cc: Tom Lane , pgsql-hackers@lists.postgresql.org Subject: Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward In-Reply-To: Message-ID: References: <2edb7a57-b225-3b23-a680-62ba90658fec@gmx.net> <1282067.1744651649@sss.pgh.pa.us> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Provags-ID: V03:K1:7cWbnXZQVZiZPN4TX7eRIh0L792lz+GYmPxuP3QI8wU+pwXL3YB HHhUEbIC2eBepZQ1+e70mdtowhpH101+1CQ/sdZgXEv6D6jECX5ffEILASVmpyZpRXEukwj TFzkIcNLhLDXO7mZe9iiWzBoQZ82M6XD9b5XKBaglWXdBQUMIey2bpTF81v0CVZytsHcrQg mLV0EHcucG/5q2TNyKSvg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:WI2j3K2D6fc=;mzUBI5k2I3HJgs2lmkFJ3GEtiwp ngO0NguHSHJFHhYr3jYaFEFn56hy7EBjxeb6yxiZvkJTOB+QL7aenGKGgyjzxMi00QgNVicSw u9cnWvmPOy1n0tzAUQGx3hPqXmoT4YIXOKcRdkR+tCPPS+UDJxeJc3QBfGo6bGEs6n/PMVcgG 9FlfBlAXKhI8eoy145jFXzBAQYYUlQPIAyFeOY0yJITJk5yUZ/m+p+9E50hPRh1LgqJMfNNwD jPnV54sCMZcvoRsYIgcm7B7NzrPTZKl581KEs7mm+hevrTM8SKZvbfGjBdZhVkfxxhXmoTwXJ UCRHnK9zH34CZ/Bfe1VaNp5fq96ALcpNgnzt8eQyNAnHegEmDS+T6Syv1CohI0++bs9MJcTdf /Xjq9wkWUmxYY94axX1O1292v98Vl/27myQklTJDE1oskjEJKmKazMxP2DX21SG2Ju+V56SF3 vT39cQ75CJQgguGT2gxl3PQxfUFKBzVBPSEHZ5rI2GgO4jxauv/SjcH9UtPL7R0WUniNThLNJ gV2jIJ8JIyhZcCD8jIxKBmR3TRbtNIdINKMGJ3iKcM0NmFbCR98N4AFd2LS/UaCpLZFKt8zjR x7Vk2GBjmdhJxbZc+nymV7ZAO7Q5Ty96TX7hgcWayLMYsL00Y4l6mPNQqmxd9OoZMxRXphjMi VfcmdiO8PmvSqfN+O0kTyW5WP26yozkOpAWLTlIGOBwCbqWSFAm//wO5cjcJiT9PmJgTPS9Md xy4FlXwC2OyHg0B4iiWFaBNfMcj/zGE4V7OxrBYuceisEB37kjkBpKSp1YHODtPQdp+wgNQSI iLStbhP+B7tIjwmLQBN5VEKXbQ68bQbFeinxNo+7fo2GNWEQwDSAmyEoNfP805Zdtv6a/TXm4 Ho7e9QTKzuG8QCGhWDx+RJW6PxbT1+DB+gAw9+NB8z6WB8qLS2WZn0ZLKBywq0DEZDPdezNFl l/WwTUZb4tkE3Is5Gz/vscrPOjNRqpc361MXrdzmqrWpqhECrxcan6mc9bka5WoG347WHvgJp K2dt2jnrBwswnH9s3OgpbZtcx80Pft2oJWWRebs7JsaQr2HDBf0C8i7Af4FvXlHp7Tei+beXO TXoKX8URv0pTvgq1cbGNeO5tymDd/p5TukwXOdKM+g12682QV4/N9BXi68/e9psSZmLgWV093 4oXg0mB3GzICt7DcZdKfcpRwv5PXAIzQEwu62/pG5aWzLT0LaIuiwKbsWe7vVlEEPN2vNNthk kJKo/ThuaB8qCsQbzN/NTStMdmeDOSqyeWzrhoMfyCn0dt/Qt5MAaPE1oKrejh6rRryDeEYHo W1eGYKUMXGfTz1iRVC/WTsuOVGAdf1brLwlmAu/nja23smK6Ai3sMLYMwGXzDqo6lbxsrmWYg 4GO7bkxxwpcbpIbrHA1BC0o50YeCJjFPAu/WM7lnXjdkLRRq70OwwVFMZPMktn2hX9sUFQK8X o0sTL5iplHlv1jUxVTegv8ytSx0q01yFF5GiEFdOWcxgabi9TtkqMPCZ25iaBpBkN5y8lHLev BZsjbVRGtYQB+Jtbmk9kYwTayPs0tRBtZIKF0KeoEO3PuK4CmBVuweHnxVVEiEirDqW+MwoxH FRlOzCrrCzpjWOzz4DfBkYqAp+h90cxFivtQhirzneKsmiF5cbbTzLg1kUd9u5kRelzPj3aUi BsT4VuYzH5t4OPM4slsNKdfuZ4OKYgJKEToKnUxSipTNyai8MBKrpPAU0tiXke+2fx0GIIpk8 Qu2/NPDlEcIWJftNwWz0O/xYnrq2nbt8NJ09dam4gWAcpNyhEWNWMokdAndhJV/V3uXH03CHV HM+TL+RDisVt1xqqWD/QimI1DNfJ0CxEBRILoRTz7d07XhWiv0taed1cq020UjXBzkU3n3bJn OISq93oNWQU+xUdglW3GwkUaj3vLMQo35UW3Av8jiUD8L/5BPysFZNUOx3gqYi3UeIKi9Vlmk kWn/FNVmOCXNL02LD6cB6QuEzEIK1vouPPNVsMk+uBGX4oFNkxk6BoZ2rZdoL4IlHXZUo3h9S 3+qWT0xodzCyXLsbS909GrvHf8UrUvUACUgnfTrJzlzryWZMGGyq0r9/JZ+rs6Oldt+hoLdse Aefj9YFEE0ie+YEof5h0Ow7Pg2RZp8Ev9cM7FNrKAhcoHyAbsGZH+Oo4f+hoZJ3/OzDkC1uaB cXFEU6A67N1QYikhy6rJn/lzlczsOorDnYg82K7AlQ5HfhgUySX62Wxwp1ioTXkAfaaUonpPH JOVEN1s0BISAsMfaoW9tUTNXR+X1u8L5dpads/OUmLu1kXm+bcb8wOMt+SzvDdxm3qO7PkG1T yTqBNDRi9Yyq3wo8xFCeCKeYxxPBL97VhN5ipw28dMyi7jSzl2i7x+FGTjBN6zLCWsNLrGO46 lupb7oiTrMSXx4FSI3mNDSFuqpGThHdD3rXMJii3KIiAvIDFEF+ulcC5lTVICi8ZwyRqPnpT0 Jgn1ol7gT5O0hluWmr5FiYlQRQmLiS1AAijOzuTGxVdK8z539Yj+Mp7tkdbnXdE8aqNsm0Cel aojD3ROJy0YCyR9x4pCL1QrvH+cB3RYR5PkJypJHsJ9PKkcHANt5zaGX6fZ1lGP0J4fsE49ch TnPRW65UVSmc7qxJLKOFITrQyZ1dap+wDNu5idOMdwznY7K8YfR+bb3m6G1mcUVhbB54o/bbr c66dUmpm9hW/0Ch1mWVSyAhlbt/rvEUvLbDf1LlthxdDJasxF7FvsdNIvIig6HYAgzmvwv6Mp iaQQ5xY8l9ig8wJnBXk2NFY3IjFhl3Jilo2uZa/b3pdDEb2OY3IhJ7aLD7B8YDrljIdCfcglN 7TRsSEfRMabx0mShqYQn5xvKViaHGaubAPBftp+/GKgck8F7MyXvvOvziomQvqYneSuIoKzyo nNDtxHSNTDtEE9NM4q3I6sl6yeK/4qncXWQAwebLTRWTjO7NPuswDT+MHhvZiD8jvITShcme7 jTzJXEvuiN9WmL2HH1rm10E5QIQit5JBnpBQgiqk4cAUTmR5GmI0GEOgyH34p4CboZ7X0QFze xjMHj2N4j4pa4S9MWGJfpDy6xOEAfTlnAoGRUueV120XgcOQFJG4NmIMP6yvloz+RsPPFMn5K 4WRwa+ZFckCEj8JMlTqWHEsy3JSY6kWZ5zaat/shbrD8A7k8ELzUZgc84fcVoQHCn7bisaGmO JlOLzgdXfbEMh9jvnD0mRCnBuICGwGopeZJkB6bh8UC4CtRUqTQ2uR9cydqp5I4IfY14BZKMC TJcD0mJnfU+2bLSq Content-Transfer-Encoding: quoted-printable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Tue, 10 Jun 2025, Nathan Bossart wrote: > I also wrote a couple of test programs to show the difference between > fseeko-ing and fread-ing through a file with various sizes. On a Linux > machine, I see this: > > log2(n) | fseeko | fread > ---------+---------+------- > 1 | 109.288 | 5.528 > 2 | 54.881 | 2.848 > 3 | 27.65 | 1.504 > 4 | 13.953 | 0.834 > 5 | 7.1 | 0.49 > 6 | 3.665 | 0.322 > 7 | 1.944 | 0.244 > 8 | 1.085 | 0.201 > 9 | 0.658 | 0.185 > 10 | 0.443 | 0.175 > 11 | 0.253 | 0.171 > 12 | 0.102 | 0.162 > 13 | 0.075 | 0.13 > 14 | 0.061 | 0.114 > 15 | 0.054 | 0.1 > > So, fseeko() starts winning around 4096 bytes. On macOS, the difference= s > aren't quite as dramatic, but 4096 bytes is the break-even point there, > too. I imagine there's a buffer around that size somewhere... Thank you for benchmarking! Before answering in more depth, I'm curious,= =20 what read-seek pattern do you see on the system call level (as=20 shown by strace)? In pg_restore it was a constant loop of=20 read(4K)-lseek(8-16K). Dimitris