From: Stephen Frost <sfrost@snowman.net>
To: Aaron Turner <synfinatic@gmail.com>
Cc: pgsql-performance <pgsql-performance@postgresql.org>
Subject: Re: large dataset with write vs read clients
Date: Thu, 7 Oct 2010 15:00:06 -0400
Message-ID: <20101007190006.GA26232@tamriel.snowman.net> (raw)
In-Reply-To: <AANLkTimNJc=0ffgRO_gO6Vsq06=aqBYXZouhbv0kBe-i@mail.gmail.com>
References: <AANLkTimNJc=0ffgRO_gO6Vsq06=aqBYXZouhbv0kBe-i@mail.gmail.com>
* Aaron Turner (synfinatic@gmail.com) wrote:
> The graphing front end CGI is all SELECT. There's 12k tables today,
> and new tables are created each month.
That's a heck of alot of tables.. Probably more than you really need.
Not sure if reducing that number would help query times though.
> The number of rows per table
> is 100-700k, with most in the 600-700K range. 190GB of data so far.
> Good news is that queries have no joins and are limited to only a few
> tables at a time.
Have you got indexes and whatnot on these tables?
> Basically, each connection is taking about 100MB resident. As we need
> to increase the number of threads to be able to query all the devices
> in the 5 minute window, we're running out of memory. There aren't
> that many CGI connections at anyone one time, but obviously query
> performance isn't great, but honestly is surprisingly good all things
> considered.
I'm kind of suprised at each connection taking 100MB, especially ones
which are just doing simple inserts.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 196B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: sfrost@snowman.net, synfinatic@gmail.com
Subject: Re: large dataset with write vs read clients
In-Reply-To: <20101007190006.GA26232@tamriel.snowman.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox