Message-ID: <4C36D537.1090709@postnewspapers.com.au>
Date: Fri, 09 Jul 2010 15:52:23 +0800
From: Craig Ringer <craig@postnewspapers.com.au>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.9) Gecko/20100423 Lightning/1.0b1 Thunderbird/3.0.4
MIME-Version: 1.0
To: Tom Lane <tgl@sss.pgh.pa.us>
CC: Samuel Gendler <sgendler@ideasculptor.com>,
	pgsql-performance@postgresql.org
Subject: Re: Need help in performance tuning.
References: <AANLkTiknhZhKtGDidBsMWtPJpRgln9DrrsCebCEYk5XE@mail.gmail.com>
	<4C369346.6010501@postnewspapers.com.au>
	<AANLkTin4Q4APRnbJ85FRasU7n8LuhKibKYcGnexsEbTt@mail.gmail.com>
	<8760.1278650566@sss.pgh.pa.us>
In-Reply-To: <8760.1278650566@sss.pgh.pa.us>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On 09/07/10 12:42, Tom Lane wrote:
> Samuel Gendler <sgendler@ideasculptor.com> writes:
>> On Thu, Jul 8, 2010 at 8:11 PM, Craig Ringer
>> <craig@postnewspapers.com.au> wrote:
>>> If you're not using a connection pool, start using one.
> 
>> I see this issue and subsequent advice cross this list awfully
>> frequently.  Is there in architectural reason why postgres itself
>> cannot pool incoming connections in order to eliminate the requirement
>> for an external pool?
> 
> Perhaps not, but there's no obvious benefit either.  Since there's
> More Than One Way To Do It, it seems more practical to keep that as a
> separate problem that can be solved by a choice of add-on packages.

Admittedly I'm relatively ignorant of the details, but I increasingly
think PostgreSQL will need a different big architectural change in the
coming years, as the typical performance characteristics of machines
change:

It'll need to separate "running queries" from "running processes", or
start threading backends, so that one way or the other a single query
can benefit from the capabilities of multiple CPUs. The same separation,
or a move to async I/O, might be needed to get one query to concurrently
read multiple partitions of a table, or otherwise get maximum benefit
from high-capacity I/O subsystems when running just a few big, expensive
queries.

Otherwise I'm wondering if PostgreSQL will begin really suffering in
performance on workloads where queries are big and expensive but there
are relatively few of them running at a time.

My point? *if* I'm not full of hot air and there's some truth to my
blather above, any change like that might be accompanied by a move to
separate query execution state from connection state, so that idle
connections have a much lower resource cost.

OK, that's my hand-waving for the day done.

--
Craig Ringer