Date: Thu, 04 Nov 2010 14:42:08 +0000
From: Nick Matheson <Nick.D.Matheson@noaa.gov>
Subject: Re: Simple (hopefully) throughput question?
In-reply-to: <4CD19E84.7070204@squeakycode.net>
To: lists@peufeu.com
Cc: pgsql-performance@postgresql.org
Message-id: <4CD2C640.8030303@noaa.gov>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
User-Agent: Thunderbird 2.0.0.24 (X11/20101005)
References: <4CD1853F.2010800@noaa.gov> <4CD19E84.7070204@squeakycode.net>

Pierre-

Reading from the tables is very fast, what bites you is that postgres 
has to convert the data to wire format, send it to the client, and the 
client has to decode it and convert it to a format usable by your 
application. Writing a custom aggregate in C should be a lot faster 
since it has direct access to the
data itself. The code path from actual table data to an aggregate is 
much shorter than from table data to the client...


I think your comments really get at what our working hypothesis was, but 
given that our experience is limited compared to you all here on the 
mailing lists we really wanted to make sure we weren't missing any 
alternatives. Also the writing of custom aggregators will likely 
leverage any improvements we make to our storage throughput.

Thanks,

Nick