public inbox for [email protected]
help / color / mirror / Atom feedFrom: Chengpeng Yan <[email protected]>
To: Dean Rasheed <[email protected]>
Cc: Tom Lane <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Subject: Re: [PATCH] Fix overflow and underflow in regr_r2()
Date: Fri, 29 May 2026 04:22:55 +0000
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAEZATCWwqMUHnjNzBday-fh15oPNqec3vLnqj7A8-G+vnmQ0RQ@mail.gmail.com>
References: <[email protected]>
<CAEZATCUaV+qmBCh0zv0EPdBvhE2skHCzkRw78VNjBMCX9Z7h+w@mail.gmail.com>
<[email protected]>
<[email protected]>
<CAEZATCXpUwijoE2imbwWWraG63SmzgXyE+B8eoFNennu87-=kw@mail.gmail.com>
<[email protected]>
<CAEZATCWDfB8QQeqrhD2_BvDrPOF5b=nskOWaRmzMGvyntv5Tnw@mail.gmail.com>
<[email protected]>
<CAEZATCUAu5JOJnyy5yXTGsfSRCDnbrGfHCWRxYVPtxROzKH30A@mail.gmail.com>
<[email protected]>
<CAEZATCWwqMUHnjNzBday-fh15oPNqec3vLnqj7A8-G+vnmQ0RQ@mail.gmail.com>
Hi,
> On May 28, 2026, at 20:37, Dean Rasheed <[email protected]> wrote:
>
> [Shrug] I think dy is common enough to denote a difference in
> y-values, and it seems clear enough, given the large comment above it.
Makes sense, thanks for the clarification.
>> 3. Do we need to add tests for the underflow path, and perhaps for the
>> Inf/NaN guard?
>
> Yeah, I think it makes sense to include a test with underflow, since
> that really can lead to a large relative error. I don't think it's
> worth testing the Inf/NaN guard, since that's more about avoiding
> operating on technically uninitialised variables, and I don't believe
> that it actually affects the results.
>
> I've add this test case:
>
> SELECT regr_intercept(y, x) FROM (VALUES (-1e-131, 0), (2e-131,
> 3e-131)) v(x, y);
>
> Here, directly computing Sx * Sxy / Sxx causes an underflow to zero,
> while the correct result should be 1e-131. Since Sy is 3e-131, this
> makes a noticeable difference to the final result (without the patch,
> it returns an intercept of 1.5e-131, whereas with the patch, it
> correctly returns 1e-131).
>
> If there are no objections from the RMT, I'll push both of these (to
> HEAD only) in a couple of days or so.
>
> Regards,
> Dean
> <v2-0001-Improve-overflow-underflow-handling-in-regr_r2.patch><v2-0002-Improve-overflow-underflow-handling-in-regr_inter.patch>
v2 LGTM.
I ran the regression tests locally on Apple Silicon as well, and all
tests passed.
--
Best regards,
Chengpeng Yan
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: [PATCH] Fix overflow and underflow in regr_r2()
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox