From: Tom Lane <tgl@sss.pgh.pa.us>
To: eric.cyr@gmail.com
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT
Date: Mon, 22 Oct 2018 13:27:58 -0400
Message-ID: <83525.1540229278@sss.pgh.pa.us> (raw)
In-Reply-To: <15449-1cf737dd5929450e@postgresql.org>
References: <15449-1cf737dd5929450e@postgresql.org>
=?utf-8?q?PG_Bug_reporting_form?= <noreply@postgresql.org> writes:
> SELECT * FROM test_file_fdw_program_limit LIMIT 0;
> /*
> [38000] ERROR: program "echo "test"" failed Detail: child process exited
> with exit code 1
> */
> LIMIT 0 is the easiest way we found to reproduce the error, but we
> encountered same issue with LIMIT (< file lines) on a program doing an if
> exists + file output stream manipulation.
Yeah, I can reproduce this on macOS as well as Linux. Capturing stderr
shows something pretty unsurprising:
sh: line 1: echo: write error: Broken pipe
I get similar behavior from "cat"; didn't try anything else.
So the called program is behaving in a somewhat reasonable way: it's
detecting EPIPE on its stdout (after we close the pipe), reporting that,
and doing exit(1).
Unfortunately, it's not clear what we could do about that, short of
always reading the whole program output, which is likely to be a
cure worse than the disease. If the program were failing thanks to
SIGPIPE, we could recognize that as a case we can ignore ... but with
behavior like this, I don't see a reliable way to tell it apart from
a generic program failure, which surely we'd better report.
The only idea that comes to mind is to introduce a file_fdw option
specifying "read program input to end, even if query doesn't want
it all". Not quite sure which choice should be the default; you
could make an argument either way I think.
Curiously, if you do stuff like "cat longfile | head -10", you don't
get any visible error, though surely cat must be seeing the same
thing underneath ... I wonder how the shell is arranging that?
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: tgl@sss.pgh.pa.us, eric.cyr@gmail.com, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT
In-Reply-To: <83525.1540229278@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox