From: Dimitrios Apostolou <jimis@gmx.net>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward
Date: Wed, 11 Jun 2025 00:32:58 +0200 (CEST)
Message-ID: <c8d94212-72cc-cd9f-162f-8eb845ba104f@gmx.net> (raw)
In-Reply-To: <aEioDuePEBRnfJYk@nathan>
References: <2edb7a57-b225-3b23-a680-62ba90658fec@gmx.net>
<a7344d8f-927c-1f97-6cc0-ac630deb98c4@gmx.net>
<c0e531f9-accc-3a93-e9e6-2cec1471f9e9@gmx.net>
<1282067.1744651649@sss.pgh.pa.us>
<c6bb93c1-4042-743c-096c-27c5e7c73347@gmx.net>
<f7a23708-c336-c189-f155-4b3197bf6ea0@gmx.net>
<aEioDuePEBRnfJYk@nathan>
On Tue, 10 Jun 2025, Nathan Bossart wrote:
> I also wrote a couple of test programs to show the difference between
> fseeko-ing and fread-ing through a file with various sizes. On a Linux
> machine, I see this:
>
> log2(n) | fseeko | fread
> ---------+---------+-------
> 1 | 109.288 | 5.528
> 2 | 54.881 | 2.848
> 3 | 27.65 | 1.504
> 4 | 13.953 | 0.834
> 5 | 7.1 | 0.49
> 6 | 3.665 | 0.322
> 7 | 1.944 | 0.244
> 8 | 1.085 | 0.201
> 9 | 0.658 | 0.185
> 10 | 0.443 | 0.175
> 11 | 0.253 | 0.171
> 12 | 0.102 | 0.162
> 13 | 0.075 | 0.13
> 14 | 0.061 | 0.114
> 15 | 0.054 | 0.1
>
> So, fseeko() starts winning around 4096 bytes. On macOS, the differences
> aren't quite as dramatic, but 4096 bytes is the break-even point there,
> too. I imagine there's a buffer around that size somewhere...
Thank you for benchmarking! Before answering in more depth, I'm curious,
what read-seek pattern do you see on the system call level (as
shown by strace)? In pg_restore it was a constant loop of
read(4K)-lseek(8-16K).
Dimitris
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: jimis@gmx.net, nathandbossart@gmail.com, tgl@sss.pgh.pa.us, pgsql-hackers@lists.postgresql.org
Subject: Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward
In-Reply-To: <c8d94212-72cc-cd9f-162f-8eb845ba104f@gmx.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox