Message-ID: <4CEC5830.90805@postnewspapers.com.au>
Date: Wed, 24 Nov 2010 08:11:28 +0800
From: Craig Ringer <craig@postnewspapers.com.au>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc14 Thunderbird/3.1.4
MIME-Version: 1.0
To: Ivan Voras <ivoras@freebsd.org>
CC: pgsql-performance@postgresql.org
Subject: Re: Performance under contention
References: <4CE9773D0200002500037CA6@gw.wicourts.gov>
	<iccjub$bn0$1@dough.gmane.org>
	<4CEA373D0200002500037CED@gw.wicourts.gov>
	<ice2pd$ae8$1@dough.gmane.org>
In-Reply-To: <ice2pd$ae8$1@dough.gmane.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 11/22/2010 11:38 PM, Ivan Voras wrote:
> On 11/22/10 16:26, Kevin Grittner wrote:
>> Ivan Voras<ivoras@freebsd.org> wrote:
>>> On 11/22/10 02:47, Kevin Grittner wrote:
>>>> Ivan Voras wrote:
>>>>
>>>>> After 16 clients (which is still good since there are only 12
>>>>> "real" cores in the system), the performance drops sharply
>>>>
>>>> Yet another data point to confirm the importance of connection
>>>> pooling. :-)
>>>
>>> I agree, connection pooling will get rid of the symptom. But not
>>> the underlying problem. I'm not saying that having 1000s of
>>> connections to the database is a particularly good design, only
>>> that there shouldn't be a sharp decline in performance when it
>>> does happen. Ideally, the performance should remain the same as it
>>> was at its peek.
>>
>> Well, I suggested that we add an admission control[1] mechanism,
>
> It looks like a hack (and one which is already implemented by connection
> pool software); the underlying problem should be addressed.

My (poor) understanding is that addressing the underlying problem would 
require a massive restructure of postgresql to separate "connection and 
session state" from "executor and backend". Idle connections wouldn't 
require a backend to sit around unused but participating in all-backends 
synchronization and signalling. Active connections over a configured 
maximum concurrency limit would queue for access to a backend rather 
than fighting it out for resources at the OS level.

The trouble is that this would be an *enormous* rewrite of the codebase, 
and would still only solve part of the problem. See the prior discussion 
on in-server connection pooling and admission control.

Personally I think the current approach is clearly difficult for many 
admins to understand and it's unfortunate that it requires external 
software to be effective. OTOH, I'm not sure what the answer is.

--
Craig Ringer