From: Scott Marlowe <smarlowe@g2switchworks.com>
To: Benjamin Krajmalnik <kraj@illumen.com>
Cc: Ben K. <bkim@coe.tamu.edu>
Cc: pgsql-admin@postgresql.org
Subject: Re: New system recommendations
Date: Fri, 28 Apr 2006 10:24:16 -0500
Message-ID: <1146237856.23538.352.camel@state.g2switchworks.com> (raw)
In-Reply-To: <8511B4970E0D124898E973DF496F9B432515DE@stash.stackdump.local>
References: <8511B4970E0D124898E973DF496F9B432515DE@stash.stackdump.local>
I'm putting both the private email and the thread back on the list, as
there's interesting data in here people could use. I don't think I'm
betraying any trust here, but if I am, please, let me know...
On Thu, 2006-04-27 at 17:37, Benjamin Krajmalnik wrote:
> Thanks for the feedback.
> I wonder if the Dell backplane will take a stabdard RAID controller
> instead of using the ROMB.
> I may investigate just getting 146GB drives for the DL360 and using that
> instead, and maybe setting up the dell as a backup server, creating
> backups and moving the files over.
If you've got some spare smaller drives laying about, you can always set
up both and benchmark them against each other, plus that gives you a
ready to go backup machine.
> Quick question - I do not have a transaction frame explicitly declared
> in m stored procedure (which is used to handle all of the inserts).
> In plpgsql, how would I wrap the code inside a transaction frame?
All stored procs run as a single transaction. I.e. the simple act of
making it a user defined function has made it a single transaction, with
all the benefits that entails.
> This is such a dilemma - we are launching a new service, but until we
> sign up clients on it I don't have the budget to go crazy on hardware.
> I must make do with what I have. What I was thinking was mybe setting
> up the Dell to be the web server and the backup database server, and
> having the production database run on the DL 360, provided I can get
> better database performance. If I can get significant better
> performance on FreeBSD I will go to it, otherwise I will stay with
> Windows.
I'd test the two. I'm guessing that at least for the initial ramp up,
windows will be ok. The real performance issue for windows is opening
connections is much slower. If the majority of your machine's time is
spent on the query, the overhead of opening connections will be lost in
the noise.
> The stored procedure is a plpgsql function which gets passed parameters
> from the monitoring agent.
> It then dynamically creates a device record for the monitored device if
> one does not yet exist.
> Once that is done it creates a test record for the particular test if
> one does not exist.
Up to now, fine. Performance on any machine should be ok.
> Once that is done, it aggregates dynamically the data into 4 tables - a
> daily snapshot, a weekly snapshot, and a monthly snapshot, and a
> dashboard snapshot.
Are you running aggregate functions against the whole table to do this?
If so, this isn't going to scale. If you're just taking the data
entered in this stored proc and adding it to a table that contains that
data, then maybe it's ok. But if it's updating based on a huge amount
of data, then that's going to be slow.
> Once all of that has been accomplished, it creates a raw log entry
> (which as of next week will go to a given partition). This data goes
> into the partitioned table to facilitate purging of retention periods
> without hammering at the database and having to rebalance indices (I
> just truncate the partition once it is no longer needed).
Sounds reasonable.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: smarlowe@g2switchworks.com, kraj@illumen.com, bkim@coe.tamu.edu
Subject: Re: New system recommendations
In-Reply-To: <1146237856.23538.352.camel@state.g2switchworks.com>
References: <8511B4970E0D124898E973DF496F9B432515DE@stash.stackdump.local> <1146237856.23538.352.camel@state.g2switchworks.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox