From: Nick Matheson <Nick.D.Matheson@noaa.gov>
To: Pierre C <lists@peufeu.com>
Cc: Andy Colson <andy@squeakycode.net>
Cc: Dan Schaffer <Daniel.S.Schaffer@noaa.gov>
Cc: Jim Nasby <jim@nasby.net>
Cc: pgsql-performance@postgresql.org
Subject: Re: Help with bulk read performance
Date: Fri, 17 Dec 2010 13:51:27 +0000
Message-ID: <4D0B6ADF.7050303@noaa.gov> (raw)
In-Reply-To: <op.vntbn2ujeorkce@apollo13>
References: <4CCECB69.7080906@noaa.gov>
<4D078CD7.7030308@squeakycode.net>
<B57C829F-DFE3-4DC7-877B-14D5B05AB4BB@nasby.net>
<4D07928B.2070907@squeakycode.net>
<4D0921D2.2030806@noaa.gov>
<op.vntbn2ujeorkce@apollo13>
Pierre-
I agree with your observation of float <-> text conversion costs, but in
this case Dan is talking about storing the raw float data (ie: 4 bytes
per float) in a bytea array so there is only the conversion from java
float[n] to java byte[4*n] which is not nearly as costly as float <->
text conversion (especially if you leave it in architecture byte order).
Nick
>
>> If the data are stored as a byte array but retrieve into a ResultSet,
>> the unpacking time goes up by an order of magnitude and the
>> observed total throughput is 25 MB/s. If the data are stored in a
>> Postgres float array and unpacked into a byte stream, the
>> observed throughput is 20 MB/s.
>
>
> float <-> text conversions are very slow, this is in fact due to the
> mismatch between base-2 (IEEE754) and base-10 (text) floating point
> representation, which needs very very complex calculations.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: Nick.D.Matheson@noaa.gov, lists@peufeu.com, andy@squeakycode.net, Daniel.S.Schaffer@noaa.gov, jim@nasby.net
Subject: Re: Help with bulk read performance
In-Reply-To: <4D0B6ADF.7050303@noaa.gov>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox