From: Oliver Crosby <ryusei@gmail.com>
To: PFC <lists@boutiquenumerique.com>
Cc: Sven Willenberger <sven@dmv.com>
Cc: Dawid Kuroczko <qnex42@gmail.com>
Cc: Kevin Grittner <Kevin.Grittner@wicourts.gov>
Cc: jd@commandprompt.com
Cc: pgsql-performance@postgresql.org
Subject: Re: Looking for tips
Date: Tue, 19 Jul 2005 21:50:18 -0400
Message-ID: <1efd553a050719185059de2581@mail.gmail.com> (raw)
In-Reply-To: <op.st6nlincth1vuj@localhost>
References: <s2dd0700.023@gwmta.wicourts.gov>
<1efd553a0507191308219a098b@mail.gmail.com>
<758d5e7f0507191319636afaa3@mail.gmail.com>
<1efd553a050719132836c31b78@mail.gmail.com>
<1121805961.3674.25.camel@lanshark.dmv.com>
<1efd553a050719140462d41468@mail.gmail.com>
<op.st6nlincth1vuj@localhost>
Sorry for the lack of specifics...
We have a file generated as a list of events, one per line. Suppose
lines 1,2,3,5,7,11,etc were related, then the last one would specify
that it's the last event. Gradually this gets assembled by a perl
script and when the last event is encountered, it gets inserted into
the db. For a given table, let's say it's of the form (a,b,c) where
'a' is a pkey, 'b' is indexed, and 'c' is other related information.
The most common 'b' values are cached locally with the perl script to
save us having to query the db. So what we end up having is:
if 'b' exists in cache, use cached 'a' value and continue
else if 'b' exists in the db, use the associated 'a' value and continue
else add a new line with 'b', return the new 'a' and continue
The local cache was a huge time saver with mysql. I've tried making a
plpgsql function that handles everything in one step on the db side,
but it didn't show any improvement. Time permitting, I'll try some new
approaches with changing the scripts and queries, though right now I
was just hoping to tune postgresql.conf to work better with the
hardware available.
Thanks to everyone for your help. Very much appreciated.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: ryusei@gmail.com, lists@boutiquenumerique.com, sven@dmv.com, qnex42@gmail.com, Kevin.Grittner@wicourts.gov, jd@commandprompt.com
Subject: Re: Looking for tips
In-Reply-To: <1efd553a050719185059de2581@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox