Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pNkif-0004pH-JO for pgsql-committers@arkaria.postgresql.org; Fri, 03 Feb 2023 01:18:25 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pNkid-0000yd-Ha for pgsql-committers@arkaria.postgresql.org; Fri, 03 Feb 2023 01:18:23 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pNkid-0000yR-AV for pgsql-committers@lists.postgresql.org; Fri, 03 Feb 2023 01:18:23 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pNkiX-0006Ed-0Y for pgsql-committers@lists.postgresql.org; Fri, 03 Feb 2023 01:18:22 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 3131ID2i1604327; Thu, 2 Feb 2023 20:18:13 -0500 From: Tom Lane To: Dean Rasheed cc: pgsql-committers@lists.postgresql.org Subject: Re: pgsql: Ensure that numeric.c compiles with other NBASE values. In-reply-to: References: <1535541.1675350926@sss.pgh.pa.us> Comments: In-reply-to Dean Rasheed message dated "Fri, 03 Feb 2023 00:05:23 +0000" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <1604325.1675387093.1@sss.pgh.pa.us> Date: Thu, 02 Feb 2023 20:18:13 -0500 Message-ID: <1604326.1675387093@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Dean Rasheed writes: > However, looking more closely, I think this function is more or less > completely broken: > 1). It doesn't work if log10val2 < 0, because then m < 0, and it > doesn't multiply by the remainder. And it then throws an overflow > error, because result.dscale comes out wrong when m < 0. > 2). The result.dscale calculation is wrong if log10val2 is a multiple > of DEC_DIGITS, causing it to drop the last 4 digits. > 3). If the scaled-up dividend doesn't fit in an int64, the numeric > computation breaks if log10val2 >= 10 due to integer overflow. Ugh. I'm not quite sure that it's worth expending code space on the log10val2 < 0 case (compared to just "Assert(log10val2 >= 0"). On the other hand, it's not much extra code, and committing it now might save somebody reinventing that logic in future. I've not actually tested the patch, but it looks reasonable by eyeball. regards, tom lane