pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Laurenz Albe <laurenz.albe@cybertec.at>
To: Harrison Borges <harrison@rlly.com>
To: pgsql-general@lists.postgresql.org
Subject: Re: Issues Scaling Postgres Concurrency
Date: Tue, 14 Mar 2023 22:47:43 +0100
Message-ID: <b64a4d9fbc2ed8604886ba7994629f708c7d4e00.camel@cybertec.at> (raw)
In-Reply-To: <CA+gH7FFqADdxMTwh1DWMM53XeAzu3TjL80DEV28baAvtctSwdw@mail.gmail.com>
References: <CA+gH7FFqADdxMTwh1DWMM53XeAzu3TjL80DEV28baAvtctSwdw@mail.gmail.com>

On Mon, 2023-03-13 at 12:24 -0400, Harrison Borges wrote:
> I’m running into severe performance problems with Postgres as I increase the number
> of concurrent requests against my backend. I’ve identified that the bottleneck is
> Postgres, and to simplify the test case, I created an endpoint that only does a
> count query on a table with ~500k rows. At 5 concurrent users, the response time
> was 33ms, at 10 users it was 60ms, and at 20 users it was 120ms.
> 
> As the number of concurrent users increases, the response time for the count query
> also increases significantly, indicating that Postgres may not be scaling well to
> handle the increasing load. 
> 
> This manifests in essentially a server meltdown on production. As the concurrent
> requests stack up, our server is stuck waiting for more and more queries.
> Eventually requests begin timing out as they start taking over 30 seconds to respond.
> 
> Am I doing something obviously wrong? Does this sound like normal behavior?

That sounds like quite normal and expected behavior.

A query that counts the number of rows in a table of half a million rows is
quite expensive and keeps a CPU core busy for a while (provided everything is
cached). At some degree of parallelism, your CPU is overloaded, which leads
to non-linear slowdown.

The thing you are doing wrong is that you are putting too much load on this
system.

Yours,
Laurenz Albe





view thread (6+ messages)  latest in thread

Message-ID: <b64a4d9fbc2ed8604886ba7994629f708c7d4e00.camel@cybertec.at>
Permalink:  ../b64a4d9fbc2ed8604886ba7994629f708c7d4e00.camel@cybertec.at/
Also on:    postgresql.org/message-id/b64a4d9fbc2ed8604886ba7994629f708c7d4e00.camel@cybertec.at

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: laurenz.albe@cybertec.at, harrison@rlly.com
  Subject: Re: Issues Scaling Postgres Concurrency
  In-Reply-To: <b64a4d9fbc2ed8604886ba7994629f708c7d4e00.camel@cybertec.at>
  References: <CA+gH7FFqADdxMTwh1DWMM53XeAzu3TjL80DEV28baAvtctSwdw@mail.gmail.com> <b64a4d9fbc2ed8604886ba7994629f708c7d4e00.camel@cybertec.at>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox