X-Original-To: pgsql-sql-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 250A75E46E7 for ; Tue, 17 Aug 2004 14:21:29 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 74437-04 for ; Tue, 17 Aug 2004 17:21:24 +0000 (GMT) Received: from face.preventsys.com (unknown [207.158.60.2]) by svr1.postgresql.org (Postfix) with SMTP id 3A13C5E46DF for ; Tue, 17 Aug 2004 14:21:24 -0300 (ADT) Received: (qmail 17452 invoked by uid 0); 17 Aug 2004 17:21:22 -0000 Received: from unknown (HELO preventsys.com) (10.1.0.28) by face.preventsys.com with SMTP; 17 Aug 2004 17:21:22 -0000 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C4847E.9DD748D8" Subject: Possible rounding error of large float values? X-MimeOLE: Produced By Microsoft Exchange V6.0.6556.0 Date: Tue, 17 Aug 2004 10:21:22 -0700 Message-ID: <8758F8D58219684FAB0239EE8967048A010807@calculon.preventsys.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Possible rounding error of large float values? Thread-Index: AcSEfp2vEkVw0AmbS0S1oHHYsdZb5A== From: "Adam Lancaster" To: X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.7 tagged_above=0.0 required=5.0 tests=HTML_40_50, HTML_MESSAGE, HTML_TAG_BALANCE_TABLE X-Spam-Level: X-Archive-Number: 200408/255 X-Sequence-Number: 18752 This is a multi-part message in MIME format. ------_=_NextPart_001_01C4847E.9DD748D8 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable When setting a float column to this value: 9223372036854775807 It gets selected out as: 9.22337203685478E18 Which appears to be rounded. When we cast it to numeric type we get: 9223372036854780000 Which also is rounded. It is still possible to find the row using the original value select * from where =3D 9223372036854775807 Is this expected behavior? Thanks, Adam ------_=_NextPart_001_01C4847E.9DD748D8 Content-Type: text/html; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Possible rounding error of large float values?

When setting a float column to this value:=

9223372036854775807

It gets selected out as:

9.22337203685478E18

Which appears to be rounded. When we cast = it to numeric type we get:

9223372036854780000

Which also is rounded. It is still possibl= e to find the row using the original value

select * from <table> where <col&= gt; =3D 9223372036854775807

Is this expected behavior?


Thanks,
Adam

------_=_NextPart_001_01C4847E.9DD748D8--