From: Tom Lane <tgl@sss.pgh.pa.us>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Rémi Zara <remi_zara@mac.com>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: pgsql: Add kqueue(2) support to the WaitEventSet API.
Date: Thu, 20 Feb 2020 14:44:04 -0500
Message-ID: <19710.1582227844@sss.pgh.pa.us> (raw)
In-Reply-To: <17106.1582224047@sss.pgh.pa.us>
References: <E1izCmM-0005pV-Co@gemulon.postgresql.org>
<20200220072435.GM2288@paquier.xyz>
<CA+hUKG+yGQDPdwcH5euDb_pE7OcnbrvGDLoOi2JMaCwj4G+ttg@mail.gmail.com>
<08364CCB-07F5-423E-8CE0-ABF0195C4CA2@mac.com>
<17106.1582224047@sss.pgh.pa.us>
[ redirecting to -hackers ]
I wrote:
> =?utf-8?Q?R=C3=A9mi_Zara?= <remi_zara@mac.com> writes:
> Le 20 févr. 2020 à 12:15, Thomas Munro <thomas.munro@gmail.com> a écrit :
>>> Remi, any chance you could run gmake installcheck under
>>> contrib/postgres_fdw on that host, to see if this is repeatable? Can
>>> you tell us about the relevant limits? Maybe ulimit -n (for the user
>>> that runs the build farm), and also sysctl -a | grep descriptors,
>>> sysctl -a | grep maxfiles?
> I have a working NetBSD 8/ppc installation, will try to reproduce there.
Yup, it reproduces fine here. I see
$ ulimit -a
...
nofiles (-n descriptors) 128
which squares with the sysctl values:
proc.curproc.rlimit.descriptors.soft = 128
proc.curproc.rlimit.descriptors.hard = 1772
kern.maxfiles = 1772
and also with set_max_safe_fds' results:
2020-02-20 14:29:38.610 EST [2218] DEBUG: max_safe_fds = 115, usable_fds = 125, already_open = 3
It seems fairly obvious now that I look at it, but: the epoll and kqueue
variants of CreateWaitEventSet are both *fundamentally* unsafe, because
they assume that they can always get a FD when they want one, which is
not a property that we generally want backend code to have. The only
reason we've not seen this before with epoll is a lack of testing
under lots-of-FDs stress.
The fact that they'll likely leak those FDs on subsequent failures is
another not-very-nice property.
I think we ought to redesign this so that those FDs are handed out by
fd.c, which can ReleaseLruFile() and retry if it gets EMFILE or ENFILE.
fd.c could also be responsible for the resource tracking needed to
prevent leakage.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: tgl@sss.pgh.pa.us, thomas.munro@gmail.com, remi_zara@mac.com, michael@paquier.xyz, pgsql-hackers@lists.postgresql.org
Subject: Re: pgsql: Add kqueue(2) support to the WaitEventSet API.
In-Reply-To: <19710.1582227844@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox