From: Tom Lane <tgl@sss.pgh.pa.us>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
cc: gkokolatos@pm.me, pgsql-hackers@lists.postgresql.org
Subject: Re: Strange behavior with polygon and NaN
In-reply-to: <20201113.153558.1784247402374030454.horikyota.ntt@gmail.com>
References: <20201105.140732.2068435975647079587.horikyota.ntt@gmail.com>
 <YP74y35GS9uaMLpKm0BiA_eK6etwydDvLfty44ykXV_cY74D9joc1H1xOI27Bq5L6YHwEV7huDY43hwbXu5i6RWK9vL53TgsbUtdVD26cP0=@pm.me>
 <2200966.1605036608@sss.pgh.pa.us>
 <20201113.153558.1784247402374030454.horikyota.ntt@gmail.com>
Comments: In-reply-to Kyotaro Horiguchi <horikyota.ntt@gmail.com>
	message dated "Fri, 13 Nov 2020 15:35:58 +0900"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <289048.1605284781.1@sss.pgh.pa.us>
Date: Fri, 13 Nov 2020 11:26:21 -0500
Message-ID: <289049.1605284781@sss.pgh.pa.us>
Precedence: bulk

Kyotaro Horiguchi <horikyota.ntt@gmail.com> writes:
> At Tue, 10 Nov 2020 14:30:08 -0500, Tom Lane <tgl@sss.pgh.pa.us> wrote in 
>> For instance, {1,-1,0} is the line "x = y".  We could argue about
>> whether it'd be sensible to return zero for the distance between that
>> and the point (inf,inf), but surely any point with one inf and one
>> finite coordinate must be an infinite distance away from that line.
>> There's nothing ill-defined about that situation.

> Mmm... (swinging my arms to mimic lines..)
> dist(x = y, (1e300, Inf)) looks indeterminant to me..

Well, what you're showing is that we get an internal overflow,
essentially, on the way to calculating the result.  Which is true,
so it's sort of accidental that we got a sensible result before.
Nonetheless, we *did* get a sensible result, so producing NaN
instead seems like a regression.

We might need to introduce special-case handling to protect the
low-level calculations from ever seeing NaN or Inf in their inputs.
Getting the right answer to "just fall out" of those calculations
might be an unreasonable hope.

For example, for a line with positive slope (A and B of opposite
signs), I think that the right answer for points (Inf,Inf) and
(-Inf,-Inf) should be NaN, on much the same grounds that Inf
minus Inf is NaN not zero.  But all other points involving any Inf
coordinates are clearly an infinite distance away from that line.

			regards, tom lane