From: Markus Schaber <schabi@logix-tt.com>
To: pgsql-performance@postgresql.org
Subject: Re: Performace Optimization for Dummies
Date: Tue, 03 Oct 2006 10:15:27 +0200
Message-ID: <45221C1F.3010908@logix-tt.com> (raw)
In-Reply-To: <efsju4$1fvi$1@news.hub.org>
References: <efgu4v$276m$1@news.hub.org>
<451D0F67.4040309@logix-tt.com>
<efsju4$1fvi$1@news.hub.org>
Hi, Carlo,
Carlo Stonebanks wrote:
>> Did you think about putting the whole data into PostgreSQL using COPY in
>> a nearly unprocessed manner, index it properly, and then use SQL and
>> stored functions to transform the data inside the database to the
>> desired result?
>
> This is actually what we are doing. The slowness is on the row-by-row
> transformation. Every row reqauires that all the inserts and updates of the
> pvious row be committed - that's why we have problems figuring out how to
> use this using SQL set logic.
Maybe "group by", "order by", "distinct on" and hand-written functions
and aggregates (like first() or best()) may help.
You could combine all relevant columns into an user-defined compund
type, then group by entity, and have a self-defined aggregate generate
the accumulated tuple for each entity.
Markus
--
Markus Schaber | Logical Tracking&Tracing International AG
Dipl. Inf. | Software Development GIS
Fight against software patents in Europe! www.ffii.org
www.nosoftwarepatents.org
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: schabi@logix-tt.com
Subject: Re: Performace Optimization for Dummies
In-Reply-To: <45221C1F.3010908@logix-tt.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox