Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pcCUp-0002Rp-1W for pgsql-general@arkaria.postgresql.org; Tue, 14 Mar 2023 21:47:51 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pcCUn-0008FJ-Q3 for pgsql-general@arkaria.postgresql.org; Tue, 14 Mar 2023 21:47:49 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pcCUn-0008F9-BW for pgsql-general@lists.postgresql.org; Tue, 14 Mar 2023 21:47:49 +0000 Received: from mail-ed1-x535.google.com ([2a00:1450:4864:20::535]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1pcCUk-0005m7-SW for pgsql-general@lists.postgresql.org; Tue, 14 Mar 2023 21:47:48 +0000 Received: by mail-ed1-x535.google.com with SMTP id eg48so8642051edb.13 for ; Tue, 14 Mar 2023 14:47:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cybertec-at.20210112.gappssmtp.com; s=20210112; t=1678830464; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id:from:to:cc:subject:date :message-id:reply-to; bh=5qSAnFCh9nKzxFlJA9VFGIyWCYDhMbU1S8vzNjgBsLI=; b=laeS0VURGN9XrIMwFOKBfJTIBoK7ri6Rp+y7uUSSCz60U+a+wbR+XjCE/rIY59u8aW qH4FueMdpGQM+8Yr+CnlerbYo7pfANpXlJMUN/g0rRMxYcAxAVJkUEgLYvtReegfv0AD yGvKyt+U+dqr/nzmgRdFn8RxpfVWCWDJovB+HhV1xOKuIgAAS+BF2zpMHdk8aN8OJMFf NR9JqgRnxZURK5SFI+89WTdS8ZCnMQxnHb8h1WB0lh2zmzPxdvCTt9Wc+yoC4ncKVKNV iVVJSBS31qU6rHLX5DaKWmtXcJtvDe6LxlqxRYxkoH8NvB1B9EfnqeG53PbyEuzZQN6G Pg5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678830464; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=5qSAnFCh9nKzxFlJA9VFGIyWCYDhMbU1S8vzNjgBsLI=; b=xIXjl9FjFNB6VlL1tlBfEdVgi0gbm6+cdJFZepUqkS7yvJZWwjiwodUiCy51ojHr35 kfesgG8tQQD1NTH38l4SfcxjOuBamWvdx61IzW6dRKud+ZdokHTEWHhtxL4lp7xlgWv0 1xuUT13OchyPQr8HQ7Es/V9GinCKIzVe6vdVJs3iXsKTA/NWBTnloafucL931qFtt67i ve2nfog1+LxUsaf/mslFvldDby5vjNJGeLGo/RSqK14IUe68EC0e4nZ6SvZnXIvcr/N2 WVfbqPREycoTh96dClblIEKyP26BjpRvktKH3EWa1CTt3Aa5kRFRCCeVdS5qlvsI6UM6 8XOA== X-Gm-Message-State: AO0yUKW21rplTSRcL44Z2Uf7N40eULVWq/X3eWNZJlTSeYTZhxp/1rGJ 4UHgGyyFRi0rcgaWqCIEoL6DQH7R/oJ2k1XmgQqRjOZw X-Google-Smtp-Source: AK7set85u/tGpSH5uPPXgPoiKQA+x8NUCyWgPfg195efxZ25v0P9rQCoMAxBbpnw2O6UF1avBinY9Q== X-Received: by 2002:a17:907:8b12:b0:90b:208:6179 with SMTP id sz18-20020a1709078b1200b0090b02086179mr4559612ejc.28.1678830464498; Tue, 14 Mar 2023 14:47:44 -0700 (PDT) Received: from localhost.localdomain (p50992f7c.dip0.t-ipconnect.de. [80.153.47.124]) by smtp.gmail.com with ESMTPSA id u26-20020a1709064ada00b008e125ee7be4sm1636128ejt.176.2023.03.14.14.47.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Mar 2023 14:47:44 -0700 (PDT) Message-ID: Subject: Re: Issues Scaling Postgres Concurrency From: Laurenz Albe To: Harrison Borges , pgsql-general@lists.postgresql.org Date: Tue, 14 Mar 2023 22:47:43 +0100 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.46.4 (3.46.4-1.fc37) MIME-Version: 1.0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Mon, 2023-03-13 at 12:24 -0400, Harrison Borges wrote: > I=E2=80=99m running into severe performance problems with Postgres as I i= ncrease the number > of concurrent requests against my backend. I=E2=80=99ve identified that t= he bottleneck is > Postgres, and to simplify the test case, I created an endpoint that only = does a > count query on a table with ~500k rows. At 5 concurrent users, the respon= se time > was 33ms, at 10 users it was 60ms, and at 20 users it was 120ms. >=20 > As the number of concurrent users increases, the response time for the co= unt query > also increases significantly, indicating that Postgres may not be scaling= well to > handle the increasing load.=C2=A0 >=20 > This manifests in essentially a server meltdown on production. As the con= current > requests stack up, our server is stuck waiting for more and more queries. > Eventually requests begin timing out as they start taking over 30 seconds= to respond. >=20 > Am I doing something obviously wrong? Does this sound like normal behavio= r? That sounds like quite normal and expected behavior. A query that counts the number of rows in a table of half a million rows is quite expensive and keeps a CPU core busy for a while (provided everything = is cached). At some degree of parallelism, your CPU is overloaded, which leads to non-linear slowdown. The thing you are doing wrong is that you are putting too much load on this system. Yours, Laurenz Albe