pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Adrian Klaver <adrian.klaver@aklaver.com>
To: Harrison Borges <harrison@rlly.com>
To: pgsql-general@lists.postgresql.org
Subject: Re: Issues Scaling Postgres Concurrency
Date: Tue, 14 Mar 2023 10:34:20 -0700
Message-ID: <8a233037-4a87-a67d-af1c-0c0d5fdc31d5@aklaver.com> (raw)
In-Reply-To: <CA+gH7FFqADdxMTwh1DWMM53XeAzu3TjL80DEV28baAvtctSwdw@mail.gmail.com>
References: <CA+gH7FFqADdxMTwh1DWMM53XeAzu3TjL80DEV28baAvtctSwdw@mail.gmail.com>

On 3/13/23 9:24 AM, Harrison Borges wrote:
> Hello everyone.
> 
> I’m running into severe performance problems with Postgres as I increase 
> the number of concurrent requests against my backend. I’ve identified 
> that the bottleneck is Postgres, and to simplify the test case, I 
> created an endpoint that only does a count query on a table with ~500k 
> rows. At 5 concurrent users, the response time was 33ms, at 10 users it 
> was 60ms, and at 20 users it was 120ms.
> 
> As the number of concurrent users increases, the response time for the 
> count query also increases significantly, indicating that Postgres may 
> not be scaling well to handle the increasing load.
> 
> This manifests in essentially a server meltdown on production. As the 
> concurrent requests stack up, our server is stuck waiting for more and 
> more queries. Eventually requests begin timing out as they start taking 
> over 30 seconds to respond.
> 
> Am I doing something obviously wrong? Does this sound like normal 
> behavior? I'm not very experienced at DB ops so I'm not 100% sure what 
> to expect here, but I have worked as a Software Engineer for over 10 
> years and I've not encountered problems like this before.

1) https://wiki.postgresql.org/wiki/Slow_Counting

2) Are you using connection pooling?

> 
> I would appreciate any insights or advice on how to optimize Postgres 
> for high concurrency scenarios. Thank you in advance for your help!


-- 
Adrian Klaver
adrian.klaver@aklaver.com





view thread (6+ messages)  latest in thread

Message-ID: <8a233037-4a87-a67d-af1c-0c0d5fdc31d5@aklaver.com>
Permalink:  ../8a233037-4a87-a67d-af1c-0c0d5fdc31d5@aklaver.com/
Also on:    postgresql.org/message-id/8a233037-4a87-a67d-af1c-0c0d5fdc31d5@aklaver.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: adrian.klaver@aklaver.com, harrison@rlly.com
  Subject: Re: Issues Scaling Postgres Concurrency
  In-Reply-To: <8a233037-4a87-a67d-af1c-0c0d5fdc31d5@aklaver.com>
  References: <CA+gH7FFqADdxMTwh1DWMM53XeAzu3TjL80DEV28baAvtctSwdw@mail.gmail.com> <8a233037-4a87-a67d-af1c-0c0d5fdc31d5@aklaver.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox