Feedback-ID: id4a34324:Fastmail
Date: Tue, 11 Feb 2025 17:27:09 -0500
From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Tomas Vondra <tomas@vondra.me>, Jelte Fennema-Nio <postgres@jeltef.nl>, 
	PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on
 startup
Message-ID: <xvd2cyrtd4wk42ugweydxfcy3bwtaymu4gqmky5fpfcu6xia4m@qbgeq23yncch>
References: <CAGECzQQh6VSy3KG4pN1d=h9J=D1rStFCMR+t7yh_Kwj-g87aLQ@mail.gmail.com>
 <3203865.1739301613@sss.pgh.pa.us>
 <94798ef1-0f13-416a-983a-88447e434a7f@vondra.me>
 <7u7dbn6s2i6bf3hjzkbqaexj2bpoblqxwbkffbetl4rjv6dcom@s2uickjc5z53>
 <3216369.1739308717@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <3216369.1739308717@sss.pgh.pa.us>
Archived-At: <https://www.postgresql.org/message-id/xvd2cyrtd4wk42ugweydxfcy3bwtaymu4gqmky5fpfcu6xia4m%40qbgeq23yncch>
Precedence: bulk

Hi,

On 2025-02-11 16:18:37 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > And when using something like io_uring for AIO, it'd allow to
> > max_files_per_process in addition to the files requires for the io_uring
> > instances.
>
> Not following?  Surely we'd not be configuring that so early in
> postmaster start?

The issue is that, with io_uring, we need to create one FD for each possible
child process, so that one backend can wait for completions for IO issued by
another backend [1]. Those io_uring instances need to be created in
postmaster, so they're visible to each backend. Obviously that helps to much
more quickly run into an unadjusted soft RLIMIT_NOFILE, particularly if
max_connections is set to a higher value.

In the current version of the AIO patchset, the creation of those io_uring
instances does happen as part of an shmem init callback, as the io uring
creation also sets up queues visible in shmem. And shmem init callbacks are
currently happening *before* postmaster's set_max_safe_fds() call:


	/*
	 * Set up shared memory and semaphores.
	 *
	 * Note: if using SysV shmem and/or semas, each postmaster startup will
	 * normally choose the same IPC keys.  This helps ensure that we will
	 * clean up dead IPC objects if the postmaster crashes and is restarted.
	 */
	CreateSharedMemoryAndSemaphores();

	/*
	 * Estimate number of openable files.  This must happen after setting up
	 * semaphores, because on some platforms semaphores count as open files.
	 */
	set_max_safe_fds();


So the issue would actually be that we're currently doing set_max_safe_fds()
too late, not too early :/

Greetings,

Andres Freund

[1] Initially I tried to avoid that, by sharing a smaller number of io_uring
    instances across backends. Making that work was a fair bit of code *and*
    was considerably slower, due to now needing a lock around submission of
    IOs. Moving to one io_uring instance per backend fairly dramatically
    simplified the code while also speeding it up.