To: Eliot Simcoe <esimcoe@mac.com>
Cc: pgsql-interfaces@postgresql.org
Subject: Re: libpq binary transfer of the numeric data type 
In-reply-to: <405AD008-BE30-11D8-B774-000A959CE7FA@mac.com> 
References: <405AD008-BE30-11D8-B774-000A959CE7FA@mac.com>
Comments: In-reply-to Eliot Simcoe <esimcoe@mac.com>
	message dated "Mon, 14 Jun 2004 14:25:45 -0400"
Date: Mon, 14 Jun 2004 15:37:28 -0400
Message-ID: <5660.1087241848@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Eliot Simcoe <esimcoe@mac.com> writes:
> it has been a struggle using the binary I/O method supported by 
> PQexecParams() for both parameter passing and result retrieval due to 
> the lack of documentation on the internal data representations used by 
> PostgreSQL,

As of 7.4 the "binary" representation used on-the-wire is NOT
necessarily the same as the server internal representation; it is
big-endian for one thing, and the layout is simplified where possible.
The documentation is still somewhat lacking; best bet is to read the
source code for the send/recv routines for the data types you are
interested in.

> and after reading a number of posts from 1999 frowning upon 
> its usage I am aware of the issues associate with this approach.

Anything dated 1999 is not very relevant to the approach we are actually
using today.

> Could someone please explain to me what the varlen field is used for? 

varlen is not present in any of the 7.4 on-the-wire formats.  According
to numeric_recv,
	 * External format is a sequence of int16's:
	 * ndigits, weight, sign, dscale, NumericDigits.

Some other relevant comments are

 * The value represented by a NumericVar is determined by the sign, weight,
 * ndigits, and digits[] array.
 * Note: the first digit of a NumericVar's value is assumed to be multiplied
 * by NBASE ** weight.	Another way to say it is that there are weight+1
 * digits before the decimal point.  It is possible to have weight < 0.

 * dscale, or display scale, is the nominal precision expressed as number
 * of digits after the decimal point (it must always be >= 0 at present).
 * dscale may be more than the number of physically stored fractional digits,
 * implying that we have suppressed storage of significant trailing zeroes.
 * It should never be less than the number of stored digits, since that would
 * imply hiding digits that are present.  NOTE that dscale is always expressed
 * in *decimal* digits, and so it may correspond to a fractional number of
 * base-NBASE digits --- divide by DEC_DIGITS to convert to NBASE digits.

Personally I would suggest using text format instead of binary, at least
for "numeric"-type values.  There really is not enough gain from dealing
with the binary format to justify doing the work and dealing with
possible future incompatibilities.  Binary format makes sense to me for
simple integers and maybe for floats.

			regards, tom lane