From: Tom Lane <tgl@sss.pgh.pa.us>
To: Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: thomas.munro@enterprisedb.com
Cc: eric.cyr@gmail.com
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT
Date: Fri, 16 Nov 2018 18:10:41 -0500
Message-ID: <9660.1542409841@sss.pgh.pa.us> (raw)
In-Reply-To: <7865.1542407597@sss.pgh.pa.us>
References: <5BE4318F.4040002@lab.ntt.co.jp>
<20181109.143931.243136889.horiguchi.kyotaro@lab.ntt.co.jp>
<5BE552ED.4040304@lab.ntt.co.jp>
<20181112.185214.76711162.horiguchi.kyotaro@lab.ntt.co.jp>
<5BE966F4.1040702@lab.ntt.co.jp>
<5BEA9E82.1080709@lab.ntt.co.jp>
<30744.1542388851@sss.pgh.pa.us>
<31355.1542389631@sss.pgh.pa.us>
<7865.1542407597@sss.pgh.pa.us>
I wrote:
> I'm not quite sure whether the reached_eof test should be
> "if (bytesread == 0)" or "if (bytesread < maxread)". The former
> seems more certain to indicate real EOF; are there other cases where
> the fread might return a short read? On the other hand, if we
> support in-band EOF indicators (\. for instance) then we might
> stop without having made an extra call to CopyGetData that would
> see bytesread == 0. On the third hand, if we do do that it's
> not quite clear to me whether SIGPIPE is ignorable or not.
After still further thought, it seems like "if (bytesread == 0)"
is the right way to proceed. That protects us against incorrectly
setting reached_eof if the pipe is being run in unbuffered or
line-buffered mode. (Which copy.c doesn't do, at the moment,
but I'd just as soon this code didn't need to assume that.
Also, I'm not 100% convinced that Windows or other platforms
might not act that way.) In the case where we terminate early
because we saw an in-band EOF marker, we would also not set reached_eof,
and again that seems like a good outcome: if we see SIGPIPE it
must mean that the program kept sending data after the EOF marker,
and that seems like a good thing to complain about. (It's only
guaranteed to fail if the program sends more than the current pipe
buffer-ful, but I don't feel a need to add extra code to check for
nonempty buffer contents as we exit.)
So I think this version is probably good, although maybe it could
use an additional comment explaining the above reasoning.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: tgl@sss.pgh.pa.us, fujita.etsuro@lab.ntt.co.jp, horiguchi.kyotaro@lab.ntt.co.jp, thomas.munro@enterprisedb.com, eric.cyr@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT
In-Reply-To: <9660.1542409841@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox