From: Alexander Lakhin <exclusion@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-bugs@lists.postgresql.org, Etsuro Fujita <etsuro.fujita@gmail.com>
Subject: Re: BUG #17828: postgres_fdw leaks file descriptors on error and aborts aborted transaction in lack of fds
Date: Fri, 24 Nov 2023 07:00:01 +0300
Message-ID: <1adf8fee-5a9a-69c2-057a-87bbddebab9d@gmail.com> (raw)
In-Reply-To: <423731.1678381075@sss.pgh.pa.us>
References: <17828-122da8cba23236be@postgresql.org>
<423731.1678381075@sss.pgh.pa.us>
Hello,
09.03.2023 19:57, Tom Lane wrote:
> PG Bug reporting form <noreply@postgresql.org> writes:
>> The following script:
>> [ leaks a file descriptor per error ]
> Yeah, at least on platforms where WaitEventSets own kernel file
> descriptors. I don't think it's postgres_fdw's fault though,
> but that of ExecAppendAsyncEventWait, which is ignoring the
> possibility of failing partway through. It looks like it'd be
> sufficient to add a PG_CATCH or PG_FINALLY block there to make
> sure the WaitEventSet is disposed of properly --- fortunately,
> it doesn't need to have any longer lifespan than that one
> function.
Now that the leakage eliminated by 50c67c201/481d7d1c0 we still can observe
the assert-triggering half of the bug with something like that:
echo "
CREATE EXTENSION postgres_fdw;
SELECT id INTO t FROM generate_series(1,100000) id;
" | psql
ns=322 # assuming "ulimit -n" = 1024 and max_safe_fds / 3 = 329
for ((i=1;i<=$ns;i++)); do
echo "
DO \$d\$
BEGIN
EXECUTE \$\$CREATE SERVER loopback_$i FOREIGN DATA WRAPPER postgres_fdw
OPTIONS (
dbname '\$\$||current_database()||\$\$',
port '\$\$||current_setting('port')||\$\$'
)\$\$;
END;
\$d\$;
CREATE USER MAPPING FOR CURRENT_USER SERVER loopback_$i;
CREATE FOREIGN TABLE ft_$i (id int) SERVER loopback_$i OPTIONS (table_name 't');
"
done | psql >psql-1.log
(echo "BEGIN;";
for ((i=1;i<=$ns;i++)); do echo "DECLARE cursor_$i CURSOR FOR SELECT * FROM ft_$i;"; done
echo "SELECT * INTO nt FROM ft_1 UNION SELECT * FROM ft_2;"
) | psql >psql-2.log
That leads to:
2023-11-24 03:03:50.281 UTC [1053419] ERROR: epoll_create1 failed: Too many open files
2023-11-24 03:03:50.281 UTC [1053419] STATEMENT: SELECT * INTO nt FROM ft_1 UNION SELECT * FROM ft_2;
2023-11-24 03:03:50.282 UTC [1053419] ERROR: epoll_create1 failed: Too many open files
2023-11-24 03:03:50.282 UTC [1053419] WARNING: AbortTransaction while in ABORT state
TRAP: failed Assert("TransactionIdIsValid(proc->xid)"), File: "procarray.c", Line: 677, PID: 1053419
Call chains for the errors are the same as in the initial report [1].
[1] https://www.postgresql.org/message-id/17828-122da8cba23236be%40postgresql.org
Best regards,
Alexander
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: exclusion@gmail.com, tgl@sss.pgh.pa.us, etsuro.fujita@gmail.com
Subject: Re: BUG #17828: postgres_fdw leaks file descriptors on error and aborts aborted transaction in lack of fds
In-Reply-To: <1adf8fee-5a9a-69c2-057a-87bbddebab9d@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox