public inbox for [email protected]  

help / color / mirror / Atom feed

From: Joel Jacobson <[email protected]>
To: Dean Rasheed <[email protected]>
Cc: pgsql-hackers <[email protected]>
Subject: Re: Incorrect results from numeric round() and trunc()
Date: Mon, 08 Jul 2024 12:08:10 +0200
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAEZATCWhg+6pFhmmeCPw8yw4c7JqwYVs-54S4kt4Jpo7ZOmc_g@mail.gmail.com>
References: <CAEZATCXB+rDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n=xxQuhg@mail.gmail.com>
	<[email protected]>
	<CAEZATCWhg+6pFhmmeCPw8yw4c7JqwYVs-54S4kt4Jpo7ZOmc_g@mail.gmail.com>

On Mon, Jul 8, 2024, at 11:45, Dean Rasheed wrote:
> On Mon, 8 Jul 2024 at 00:40, Joel Jacobson <[email protected]> wrote:
>>
>> On Sun, Jul 7, 2024, at 13:28, Dean Rasheed wrote:
>> > I've also tidied up a bit by replacing all instances of SHRT_MAX with
>> > a new constant NUMERIC_WEIGHT_MAX, whose name more accurately
>> > describes the limit, as used in various other overflow checks.
>>
>> Having thought a bit more on this, I think we probably need a
>> DEC_DIGITS sensitive definition of NUMERIC_WEIGHT_MAX,
>> since per spec the max range for numeric is 0x20000 (131072)
>> decimal digits.
>>
>
> No, the maximum weight is determined by the use of int16 to store the
> weight. Therefore if you did reduce DEC_DIGITS to 1 or 2, the number
> of decimal digits allowed before the decimal point would be reduced
> too.

OK, that can actually be seen as a feature, especially since it's
of course more likely DEC_DIGITS could increase in the future
than decrease.

For example, let's say we would double it to 8,
then if NUMERIC_WEIGHT_MAX would still be 0x7FFF (32767),
then the maximum range for numeric would increase from 131072 to 262144
decimal digits allowed before the decimal point.

LGTM.

Regards,
Joel

view thread (3+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: Incorrect results from numeric round() and trunc()
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox