Date: Fri, 3 Jan 2020 09:43:40 +0100 (CET)
From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
cc: Vik Fearing <vik.fearing@2ndquadrant.com>,
    Chapman Flack <chap@anastigmatix.net>,
    PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Greatest Common Divisor
In-Reply-To: 
 <CAEZATCX9YgkWupVFcJ5ixkEJMyf2X5ZecN8K5r12oOB8mDHzWQ@mail.gmail.com>
Message-ID: <alpine.DEB.2.21.2001030929470.13581@pseudo>
References: <adbd3e0b-e3f1-5bbc-21db-03caf1cef0f7@2ndquadrant.com>
 <alpine.DEB.2.21.1912281848540.24861@pseudo>
 <dbf86808-91fa-3262-f939-2ef3d38ec242@2ndquadrant.com>
 <alpine.DEB.2.21.1912290815520.889@pseudo>
 <5E08D947.9000902@anastigmatix.net>
 <alpine.DEB.2.21.1912291755510.14206@pseudo>
 <CAEZATCX9YgkWupVFcJ5ixkEJMyf2X5ZecN8K5r12oOB8mDHzWQ@mail.gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII; format=flowed
Precedence: bulk


> Normally gcd() returns a positive integer, and gcd(a,0) = gcd(a,a) =
> abs(a). But since abs(INT_MIN) cannot be represented as a 32-bit
> integer, both those cases should throw an integer-out-of-range error.

I'm also in favor of that option, rather than sending a negative result as 
a result.

About lcm(a, b): a / gcd(a, b) * b, at least if a & b are positive. If 
not, some thoughts are needed:-)

Returning a NUMERIC as suggested by Tom would solve the overflow problem 
by sending it back to the user who has to cast. This looks ok to me.

Maybe we could provide "int4 lcm(int2, int2)", "int8 lcm(int4, int4)", as 
ISTM that there cannot be overflows on those (eg for the later: lcm <= 
a*b, a & b are 31 non-signed bits, 62 bits are needed, 63 are available).

-- 
Fabien.