public inbox for [email protected]
help / color / mirror / Atom feedFrom: Martin Norbäck Olivers <[email protected]>
To: szy <[email protected]>
Cc: Erik Brandsberg <[email protected]>
Cc: pgsql-sql <[email protected]>
Subject: Re: Inconsistent results for division and multiplication operations
Date: Tue, 26 Nov 2024 09:13:22 +0100
Message-ID: <CALoTC6uBuw0KAC_Ys5YMRfCazXh3gdxeY9OVkpvmnexsE__FAg@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
<CAFcck8EQ3h15-P=Ndt269gE50FDMc=BmWM0iVfUuLbev5HJUbQ@mail.gmail.com>
<[email protected]>
On Mon, Nov 25, 2024 at 5:18 PM szy <[email protected]> wrote:
> If the number of significant digits in the input is not fixed, it becomes
> challenging to achieve consistent results by rounding.
> for example
> postgres=# select round(1.003/1.002*5.01,2);
> ?column?
> --------------------------
> 5.01
> (1 row)
>
> postgres=# select round(1.003*5.01/1.002,2);
> ?column?
> --------------------
> 5.02
> (1 row)
>
>
Correct. That's why you should always use numeric with the desired
precision if you want precision numbers.
for instance
select 1.003/1.002*5.01 :: numeric(10,4)
will give the same result as
select 1.003*5.01/1.002 :: numeric(10,4)
They are much slower to calculate than floating point, however, so if you
don't care about precision you can keep using just floating point.
Regards,
Martin
--
Martin Norbäck Olivers
IT-konsult, Masara AB
Telefon: +46 703 22 70 12
E-post: [email protected]
Kärrhöksvägen 4
656 72 Skattkärr
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: Inconsistent results for division and multiplication operations
In-Reply-To: <CALoTC6uBuw0KAC_Ys5YMRfCazXh3gdxeY9OVkpvmnexsE__FAg@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox