From: Tom Lane <tgl@sss.pgh.pa.us>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
cc: Oleg Ivanov <o15611@gmail.com>, Laurenz Albe <laurenz.albe@cybertec.at>,
        pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #19340: Wrong result from CORR() function
In-reply-to: <923070.1764802351@sss.pgh.pa.us>
References: <19340-6fb9f6637f562092@postgresql.org>
 <4ab9867066e9545dd1a7e835a480bb0ecbe1a00d.camel@cybertec.at>
 <CAH1GMznwE=WGJYCZU1ou9iOmNR87NQFSRdYHqE3=JQ3as4P1hQ@mail.gmail.com>
 <375068.1764696127@sss.pgh.pa.us>
 <CAEZATCU8rnyKN3z_5-osk3Bn8dtzWf9nKjTr2E16-ExXiESNrQ@mail.gmail.com>
 <434484.1764707203@sss.pgh.pa.us>
 <CAEZATCUo6skdTfRP1XM1sbMveRo1PgpjzOMZEps69Bbz6-OnKA@mail.gmail.com>
 <513345.1764717860@sss.pgh.pa.us> <531516.1764721052@sss.pgh.pa.us>
 <545890.1764725276@sss.pgh.pa.us>
 <CAEZATCXgM=_6KP2RS4ShcAMbVMrVujD+fWs0vD=uOSnD_0dNmg@mail.gmail.com>
 <923070.1764802351@sss.pgh.pa.us>
Comments: In-reply-to Tom Lane <tgl@sss.pgh.pa.us>
	message dated "Wed, 03 Dec 2025 17:52:31 -0500"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <931688.1764807066.1@sss.pgh.pa.us>
Date: Wed, 03 Dec 2025 19:11:06 -0500
Message-ID: <931689.1764807066@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/931689.1764807066%40sss.pgh.pa.us>
Precedence: bulk

I wrote:
> Poking at this, I soon found a test case where even with the separate
> sqrt() calls we'd produce a result slightly outside [-1, 1] (running
> this test over more values of x is sufficient).  So now I think we
> should do both the separate sqrt and the clamp.

Per CI results, on some platforms the roundoff error is different from
what I observe, producing a value just less than 1 rather than just
more.  That doesn't invalidate needing the clamp, but it does mean
that we can't use that test case just like that.  I'm inclined to
remove the change of extra_float_digits, but keep the test case.

			regards, tom lane