pg.ddx.io pgsql-general@postgresql.org mailing list archive
help / color / mirror / Atom feedPostgresql database terminates abruptly with toomay open files error
8+ messages / 6 participants
[nested] [flat]
* Postgresql database terminates abruptly with toomay open files error
@ 2025-01-14 12:52 Sri Mrudula Attili <sri@ebi.ac.uk>
0 siblings, 1 reply; 8+ messages in thread
From: Sri Mrudula Attili @ 2025-01-14 12:52 UTC (permalink / raw)
To: pgsql-in-general@postgresql.org
Hello Team,
We have a postgresql VDB(virtual database- Delphix) that keeps
terminating due "to too many open files".
Below are few alerts that we could see from the postgresql.log
< 2025-01-14 11:37:20.724 GMT >LOG: out of file descriptors: Too many
open files in system; release and retry
< 2025-01-14 11:37:20.724 GMT >FATAL: epoll_create1 failed: Too many
open files in system
The number of allowed openfiles at OS level are 65000. even then we see
these all these being utilised and causing the database to terminate.
Is there a way we could avoid this.
Thanks,
Sri
^ permalink raw reply [nested|flat] 8+ messages in thread
* Postgresql database terminates abruptly with too many open files error
@ 2025-01-14 12:58 Sri Mrudula Attili <sri@ebi.ac.uk>
parent: Sri Mrudula Attili <sri@ebi.ac.uk>
0 siblings, 3 replies; 8+ messages in thread
From: Sri Mrudula Attili @ 2025-01-14 12:58 UTC (permalink / raw)
To: pgsql-general
Hello Team,
We have a postgresql VDB(virtual database- Delphix) that keeps
terminating due "to too many open files".
Below are few alerts that we could see from the postgresql.log
< 2025-01-14 11:37:20.724 GMT >LOG: out of file descriptors: Too many
open files in system; release and retry
< 2025-01-14 11:37:20.724 GMT >FATAL: epoll_create1 failed: Too many
open files in system
The number of allowed openfiles at OS level are 65000. even then we see
these all these being utilised and causing the database to terminate.
Is there a way we could avoid this.
Thanks,
Sri
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql database terminates abruptly with too many open files error
@ 2025-01-14 13:39 Frank Lanitz <frank@frank.uvena.de>
parent: Sri Mrudula Attili <sri@ebi.ac.uk>
2 siblings, 0 replies; 8+ messages in thread
From: Frank Lanitz @ 2025-01-14 13:39 UTC (permalink / raw)
To: pgsql-general@lists.postgresql.org
Hello,
Have you checked something like lsof to see open file descriptors to see?
Cheers,
frank
Am 14.01.25 um 13:58 schrieb Sri Mrudula Attili:
> Hello Team,
>
> We have a postgresql VDB(virtual database- Delphix) that keeps terminating due "to too many open files".
>
> Below are few alerts that we could see from the postgresql.log
>
>
> < 2025-01-14 11:37:20.724 GMT >LOG: out of file descriptors: Too many open files in system; release and retry
>
> < 2025-01-14 11:37:20.724 GMT >FATAL: epoll_create1 failed: Too many open files in system
>
>
> The number of allowed openfiles at OS level are 65000. even then we see these all these being utilised and causing the database to terminate.
>
>
> Is there a way we could avoid this.
>
>
> Thanks,
>
> Sri
>
>
>
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql database terminates abruptly with too many open files error
@ 2025-01-14 13:39 Ron Johnson <ronljohnsonjr@gmail.com>
parent: Sri Mrudula Attili <sri@ebi.ac.uk>
2 siblings, 0 replies; 8+ messages in thread
From: Ron Johnson @ 2025-01-14 13:39 UTC (permalink / raw)
To: pgsql-general
On Tue, Jan 14, 2025 at 7:58 AM Sri Mrudula Attili <sri@ebi.ac.uk> wrote:
> Hello Team,
>
> We have a postgresql VDB(virtual database- Delphix) that keeps
> terminating due "to too many open files".
>
> Below are few alerts that we could see from the postgresql.log
>
>
> < 2025-01-14 11:37:20.724 GMT >LOG: out of file descriptors: Too many
> open files in system; release and retry
>
> < 2025-01-14 11:37:20.724 GMT >FATAL: epoll_create1 failed: Too many
> open files in system
>
>
> The number of allowed openfiles at OS level are 65000. even then we see
> these all these being utilised and causing the database to terminate.
>
>
> Is there a way we could avoid this.
>
We need more information.
--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql database terminates abruptly with too many open files error
@ 2025-01-14 14:19 Tom Lane <tgl@sss.pgh.pa.us>
parent: Sri Mrudula Attili <sri@ebi.ac.uk>
2 siblings, 1 reply; 8+ messages in thread
From: Tom Lane @ 2025-01-14 14:19 UTC (permalink / raw)
To: Sri Mrudula Attili <sri@ebi.ac.uk>; +Cc: pgsql-general
Sri Mrudula Attili <sri@ebi.ac.uk> writes:
> We have a postgresql VDB(virtual database- Delphix) that keeps
> terminating due "to too many open files".
What do you have max_connections set to, and how many actually-live
server processes are there typically?
> The number of allowed openfiles at OS level are 65000.
I'm suspecting that you either need to increase that, decrease
max_files_per_process (which defaults to 1000), or decrease
max_connections.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql database terminates abruptly with too many open files error
@ 2025-01-15 11:42 Sri Mrudula Attili <sri@ebi.ac.uk>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 8+ messages in thread
From: Sri Mrudula Attili @ 2025-01-15 11:42 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-general
Hello Tom,
The max_connections =200 and max_files_per_process =1000 as you mentioned.
So should a max_files_per_process =500 helps?
I could see from the number of connections to the databaseis not
exceeding 20. But still these 20 are causing all the damage or using up
all the openfiles quota.
Thanks,
Sri
On 14/01/2025 14:19, Tom Lane wrote:
> Sri Mrudula Attili <sri@ebi.ac.uk> writes:
>> We have a postgresql VDB(virtual database- Delphix) that keeps
>> terminating due "to too many open files".
> What do you have max_connections set to, and how many actually-live
> server processes are there typically?
>
>> The number of allowed openfiles at OS level are 65000.
> I'm suspecting that you either need to increase that, decrease
> max_files_per_process (which defaults to 1000), or decrease
> max_connections.
>
> regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql database terminates abruptly with too many open files error
@ 2025-01-19 09:01 Durgamahesh Manne <maheshpostgres9@gmail.com>
parent: Sri Mrudula Attili <sri@ebi.ac.uk>
1 sibling, 0 replies; 8+ messages in thread
From: Durgamahesh Manne @ 2025-01-19 09:01 UTC (permalink / raw)
To: Sri Mrudula Attili <sri@ebi.ac.uk>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-general
---------- Forwarded message ---------
From: Sri Mrudula Attili <sri@ebi.ac.uk>
Date: Wed, 15 Jan, 2025, 17:12
Subject: Re: Postgresql database terminates abruptly with too many open
files error
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: <pgsql-general@postgresql.org>
Hello Tom,
The max_connections =200 and max_files_per_process =1000 as you mentioned.
So should a max_files_per_process =500 helps?
I could see from the number of connections to the databaseis not
exceeding 20. But still these 20 are causing all the damage or using up
all the openfiles quota.
Thanks,
Sri
On 14/01/2025 14:19, Tom Lane wrote:
> Sri Mrudula Attili <sri@ebi.ac.uk> writes:
>> We have a postgresql VDB(virtual database- Delphix) that keeps
>> terminating due "to too many open files".
> What do you have max_connections set to, and how many actually-live
> server processes are there typically?
>
>> The number of allowed openfiles at OS level are 65000.
> I'm suspecting that you either need to increase that, decrease
> max_files_per_process (which defaults to 1000), or decrease
> max_connections.
>
> regards, tom lane
Hi Team,
What is the final conclusion for this ?
Regards,
Durga Mahesh
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql database terminates abruptly with too many open files error
@ 2025-01-19 11:06 Peter J. Holzer <hjp-pgsql@hjp.at>
parent: Sri Mrudula Attili <sri@ebi.ac.uk>
1 sibling, 0 replies; 8+ messages in thread
From: Peter J. Holzer @ 2025-01-19 11:06 UTC (permalink / raw)
To: pgsql-general@lists.postgresql.org
On 2025-01-15 11:42:30 +0000, Sri Mrudula Attili wrote:
> The max_connections =200 and max_files_per_process =1000 as you mentioned.
>
> So should a max_files_per_process =500 helps?
>
>
> I could see from the number of connections to the databaseis not exceeding
> 20. But still these 20 are causing all the damage or using up all the
> openfiles quota.
If there are 20 processes with at most 1000 open files each, that's only
20'000 open files. So where do the other 45'536 files come from?
Use something like
lsof | awk '{print $1, $2}' | uniq -c | sort -n
to find the processes with the most open files (but be aware that lsof
reports file descriptors for each thread, so any multi-threaded programs
will be vastly inflated)
hp
--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2025-01-19 11:06 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-01-14 12:52 Postgresql database terminates abruptly with toomay open files error Sri Mrudula Attili <sri@ebi.ac.uk>
2025-01-14 12:58 ` Postgresql database terminates abruptly with too many open files error Sri Mrudula Attili <sri@ebi.ac.uk>
2025-01-14 13:39 ` Re: Postgresql database terminates abruptly with too many open files error Frank Lanitz <frank@frank.uvena.de>
2025-01-14 13:39 ` Re: Postgresql database terminates abruptly with too many open files error Ron Johnson <ronljohnsonjr@gmail.com>
2025-01-14 14:19 ` Re: Postgresql database terminates abruptly with too many open files error Tom Lane <tgl@sss.pgh.pa.us>
2025-01-15 11:42 ` Re: Postgresql database terminates abruptly with too many open files error Sri Mrudula Attili <sri@ebi.ac.uk>
2025-01-19 09:01 ` Re: Postgresql database terminates abruptly with too many open files error Durgamahesh Manne <maheshpostgres9@gmail.com>
2025-01-19 11:06 ` Re: Postgresql database terminates abruptly with too many open files error Peter J. Holzer <hjp-pgsql@hjp.at>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox