From: Andy Fan <zhihuifan1213@163.com>
To: Andres Freund <andres@anarazel.de>
Cc: Nathan Bossart <nathandbossart@gmail.com>,  Justin Pryzby
 <pryzby@telsasoft.com>,  Maxim Orlov <orlovmg@gmail.com>,  Pavel Borisov
 <pashkin.elfe@gmail.com>,  "Bossart, Nathan" <bossartn@amazon.com>,  Maxim
 Orlov <m.orlov@postgrespro.ru>,  pgsql-hackers@lists.postgresql.org
Subject: Re: Pre-allocating WAL files
In-Reply-To: <vfhyldq3rtnwd4uxbb7ro2opmvbmfswvkycavmzcqb55mmo4jk@g2d2n6xnwicb>
	(Andres Freund's message of "Tue, 21 Jan 2025 11:23:06 -0500")
References: <CB15BEBD-98FC-4E72-86AE-513D34014176@amazon.com>
	<CALT9ZEG+R9ne6xTyEVSiLhZ7wJAQ1Jq-9XVZs=zbd_jQJyPTdA@mail.gmail.com>
	<CACG=ezb5Os_ohi1caZJLKsGJc-nSSqGXFrMjmKw4qN8fG_=NEg@mail.gmail.com>
	<20220301144044.GG25269@telsasoft.com>
	<20220301150704.GA1030078@nathanxps13>
	<20220317231212.GB1415284@nathanxps13>
	<20220408203003.GA1630183@nathanxps13> <87sepc4zcg.fsf@163.com>
	<Z4_C04ZJ_SymvYes@nathan> <Z4_Hmj2R8NXnAI-K@nathan>
	<vfhyldq3rtnwd4uxbb7ro2opmvbmfswvkycavmzcqb55mmo4jk@g2d2n6xnwicb>
Date: Wed, 22 Jan 2025 01:14:22 +0000
Message-ID: <87o6zz4pld.fsf@163.com>
MIME-Version: 1.0
Content-Type: text/plain
Archived-At: <https://www.postgresql.org/message-id/87o6zz4pld.fsf%40163.com>
Precedence: bulk

Andres Freund <andres@anarazel.de> writes:

Hi,

> FWIW, I've seen the fsyncs around recycling being a rather substantial
> bottleneck. To the point of the main benefit of larger segments being the
> reduction in number of fsyncs at the end of a checkpoint.  I think we should
> be able to make the fsyncs a lot more efficient by batching them, first rename
> a bunch of files, then fsync them and the directory. The current pattern
> bascially requires a separate filesystem jouranl flush for each WAL segment.

For education purpose, how to fsync files in batch? 'man fsync' tells me
user can only fsync one file each time.

int fsync(int fd);

The fsync manual seems not saying fsync on a directory would fsync all
the files under that directory.

-- 
Best Regards
Andy Fan