Subject: Re: Improving connection scalability: GetSnapshotData()
To: AJG <ayden@gera.co.nz>, pgsql-hackers@postgresql.org
References: <20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de>
 <1614447658602-0.post@n3.nabble.com>
From: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
Message-ID: <4ac9a8e1-d170-7a2c-5ab5-36cbd5946a50@postgrespro.ru>
Date: Mon, 1 Mar 2021 18:46:51 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <1614447658602-0.post@n3.nabble.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
Precedence: bulk


On 27.02.2021 20:40, AJG wrote:
> Hi,
>
> Greatly appreciate if you could please reply to the following questions as
> time allows.
>
> I have seen previous discussion/patches on a built-in connection pooler. How
> does this scalability improvement, particularly idle connection improvements
> etc, affect that built-in pooler need, if any?
>
>
> Same general question about an external connection pooler in general in
> Production? Still required to route to different servers but no longer
> needed for the pooling part. as an example.
>
>
> Many Thanks!
>

Connection pooler is still needed.
The patch for GetSnapshotData() mostly improves scalability of read-only 
queries and reduce contention for procarray lock.
But read-write upload cause contention for many other resources: 
relation extension lock, buffer locks, tuple locks and so on.

If you run pgbench at NUMA machine with hundreds of cores, then you will 
still observe significant degradation of performance with increasing 
number of connection.
And this degradation will be dramatic if you replace some non-uniform 
distribution of keys, for example Zipfian distribution.


>
>
>
> --
> Sent from: https://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html
>
>