pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
Cc: pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: Ensure that numeric.c compiles with other NBASE values.
Date: Thu, 02 Feb 2023 20:18:13 -0500
Message-ID: <1604326.1675387093@sss.pgh.pa.us> (raw)
In-Reply-To: <CAEZATCW8gXgW0tgPxPgHDPhVX71+SWFRkhnXy+TfGDsKLepu2g@mail.gmail.com>
References: <E1pNW9D-001CQv-IE@gemulon.postgresql.org>
	<1535541.1675350926@sss.pgh.pa.us>
	<CAEZATCW8gXgW0tgPxPgHDPhVX71+SWFRkhnXy+TfGDsKLepu2g@mail.gmail.com>

Dean Rasheed <dean.a.rasheed@gmail.com> writes:
> However, looking more closely, I think this function is more or less
> completely broken:

> 1). It doesn't work if log10val2 < 0, because then m < 0, and it
> doesn't multiply by the remainder. And it then throws an overflow
> error, because result.dscale comes out wrong when m < 0.

> 2). The result.dscale calculation is wrong if log10val2 is a multiple
> of DEC_DIGITS, causing it to drop the last 4 digits.

> 3). If the scaled-up dividend doesn't fit in an int64, the numeric
> computation breaks if log10val2 >= 10 due to integer overflow.

Ugh.

I'm not quite sure that it's worth expending code space on the
log10val2 < 0 case (compared to just "Assert(log10val2 >= 0").
On the other hand, it's not much extra code, and committing it now
might save somebody reinventing that logic in future.

I've not actually tested the patch, but it looks reasonable
by eyeball.

			regards, tom lane





view thread (5+ messages)  latest in thread

Message-ID: <1604326.1675387093@sss.pgh.pa.us>
Permalink:  ../1604326.1675387093@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/1604326.1675387093@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: tgl@sss.pgh.pa.us, dean.a.rasheed@gmail.com
  Subject: Re: pgsql: Ensure that numeric.c compiles with other NBASE values.
  In-Reply-To: <1604326.1675387093@sss.pgh.pa.us>
  References: <E1pNW9D-001CQv-IE@gemulon.postgresql.org> <1535541.1675350926@sss.pgh.pa.us> <CAEZATCW8gXgW0tgPxPgHDPhVX71+SWFRkhnXy+TfGDsKLepu2g@mail.gmail.com> <1604326.1675387093@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox