From: Craig James <craig_james@emolecules.com>
To: pgsql-admin@postgresql.org
Subject: Connection pooling for a mixture of lightweight and heavyweight jobs?
Date: Wed, 28 Jul 2010 15:13:57 -0700
Message-ID: <4C50ABA5.5010408@emolecules.com> (raw)
I have a question that may be related to connection pooling.
We create a bunch of high-performance lightweight Postgres clients that serve up images (via mod_perl and Apache::DBI). We have roughly ten web sites, with ten mod_perl instances each, so we always have around 100 Postgres backends sitting around all the time waiting. When a lightweight request comes in, it's a single query on an primary key with no joins, so it's very fast.
We also have a very heavyweight process (our primary search technology) that can take many seconds, even minutes, to do a search and generate a web page.
The lightweight backends are mostly idle, but when a heavyweight search finishes, it causes a burst on the lightweight backends, which must be very fast. (They provide all of the images in the results page.)
This mixture seems to make it hard to configure Postgres with the right amount of memory and such. The primary query needs some elbow room to do its work, but the lightweight queries all get the same resources.
I figured that having these lightweight Postgres backends sitting around was harmless -- they allocate shared memory and other resources, but they never use them, so what's the harm? But recent discussions about connection pooling seem to suggest otherwise, that merely having 100 backends sitting around might be a problem.
Craig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: craig_james@emolecules.com
Subject: Re: Connection pooling for a mixture of lightweight and heavyweight jobs?
In-Reply-To: <4C50ABA5.5010408@emolecules.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox