agora inbox for pgsql-bugs@postgresql.org
help / color / mirror / Atom feedFrom: Tom Lane <tgl@sss.pgh.pa.us>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
Cc: Oleg Ivanov <o15611@gmail.com>
Cc: Laurenz Albe <laurenz.albe@cybertec.at>
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #19340: Wrong result from CORR() function
Date: Wed, 03 Dec 2025 19:11:06 -0500
Message-ID: <931689.1764807066@sss.pgh.pa.us> (raw)
In-Reply-To: <923070.1764802351@sss.pgh.pa.us>
References: <19340-6fb9f6637f562092@postgresql.org>
<4ab9867066e9545dd1a7e835a480bb0ecbe1a00d.camel@cybertec.at>
<CAH1GMznwE=WGJYCZU1ou9iOmNR87NQFSRdYHqE3=JQ3as4P1hQ@mail.gmail.com>
<375068.1764696127@sss.pgh.pa.us>
<CAEZATCU8rnyKN3z_5-osk3Bn8dtzWf9nKjTr2E16-ExXiESNrQ@mail.gmail.com>
<434484.1764707203@sss.pgh.pa.us>
<CAEZATCUo6skdTfRP1XM1sbMveRo1PgpjzOMZEps69Bbz6-OnKA@mail.gmail.com>
<513345.1764717860@sss.pgh.pa.us>
<531516.1764721052@sss.pgh.pa.us>
<545890.1764725276@sss.pgh.pa.us>
<CAEZATCXgM=_6KP2RS4ShcAMbVMrVujD+fWs0vD=uOSnD_0dNmg@mail.gmail.com>
<923070.1764802351@sss.pgh.pa.us>
I wrote:
> Poking at this, I soon found a test case where even with the separate
> sqrt() calls we'd produce a result slightly outside [-1, 1] (running
> this test over more values of x is sufficient). So now I think we
> should do both the separate sqrt and the clamp.
Per CI results, on some platforms the roundoff error is different from
what I observe, producing a value just less than 1 rather than just
more. That doesn't invalidate needing the clamp, but it does mean
that we can't use that test case just like that. I'm inclined to
remove the change of extra_float_digits, but keep the test case.
regards, tom lane
view thread (24+ messages) latest in thread
Message-ID: <931689.1764807066@sss.pgh.pa.us>
Permalink: ../931689.1764807066@sss.pgh.pa.us/
Also on: postgresql.org/message-id/931689.1764807066@sss.pgh.pa.us
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: tgl@sss.pgh.pa.us, dean.a.rasheed@gmail.com, o15611@gmail.com, laurenz.albe@cybertec.at, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #19340: Wrong result from CORR() function
In-Reply-To: <931689.1764807066@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox