pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedQuery Performance in bundled requests
2+ messages / 2 participants
[nested] [flat]
* Query Performance in bundled requests
@ 2020-09-08 10:30 Dirk Krautschick <Dirk.Krautschick@trivadis.com>
0 siblings, 1 reply; 2+ messages in thread
From: Dirk Krautschick @ 2020-09-08 10:30 UTC (permalink / raw)
To: Pgsql Performance <pgsql-performance@lists.postgresql.org>
Update: Better title and format corrections
Hi %,
in order to be able to readjust the effects of the stored procedure and, if necessary, to save turnaround times, different requests can be concatenated using semicolons for bundling several statements in one request. We did some tests against a postgres cluster.
The results in terms of optimizations are as follows:
Batchsize | clients| count Queries | average s/query| comment
--------------|---------|----------------------|----------------------|-
1 | 1 | 15.86k | 2.24ms |
10 | 1 | 31.80k | 332us |
25 | 1 | 31.75k | 312us |
50 | 1 | 32.00k | 280us |
100 | 1 | 32.00k | 286us |
| | | |
1 | 2 | 57.1k | 733us | Drop to 30k after some time!!
10 | 2 | 63.6k | 323us |
25 | 2 | 63.5k | 308us |
50 | 2 | 64k | 293us |
100 | 2 | 67.2k | 290us |
| | | |
1 | 10 | 158.6k | 2.15ms |
10 | 10 | 298.9k | 383us | Drop to ~200k!!
25 | 10 | 225k | 1.16ms |
50 | 10 | 192k | 1.55ms |
100 | 10 | 201.6k | 1.44ms |
| | | |
10 | 50 | 800k | 2.2ms |
It seems to be saturated here at around 200k requests per minute, the question remains why this is so.
Does anyone has experience with something similar or are there some hints about how to optimize the postgres cluster for such bundled statements?
Thanks and best regards
Dirk
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: Query Performance in bundled requests
@ 2020-09-08 10:49 Justin Pryzby <pryzby@telsasoft.com>
parent: Dirk Krautschick <Dirk.Krautschick@trivadis.com>
0 siblings, 0 replies; 2+ messages in thread
From: Justin Pryzby @ 2020-09-08 10:49 UTC (permalink / raw)
To: Dirk Krautschick <Dirk.Krautschick@trivadis.com>; +Cc: pgsql-performance@lists.postgresql.org
On Tue, Sep 08, 2020 at 10:30:50AM +0000, Dirk Krautschick wrote:
> Update: Better title and format corrections
>
> Hi %,
>
> in order to be able to readjust the effects of the stored procedure and, if necessary, to save turnaround times, different requests can be concatenated using semicolons for bundling several statements in one request. We did some tests against a postgres cluster.
>
> The results in terms of optimizations are as follows:
>
>
> Batchsize | clients| count Queries | average s/query| comment
> --------------|---------|----------------------|----------------------|-
> 1 | 1 | 15.86k | 2.24ms |
> 10 | 1 | 31.80k | 332us |
> 25 | 1 | 31.75k | 312us |
> 50 | 1 | 32.00k | 280us |
I guess you're looking at the minimum of 280us.
; 1/(280e-6) * 60
~214285.71428571428571428571
> the question remains why this is so.
You can't expect it to go a billion times faster just by putting a billion
queries in one request, and at 50 batches it looks like you've hit the next
performance bottleneck. Whether that's CPU / IO / network / locks / RAM /
planner / context switches / logging / ??? remains to be seen.
> Does anyone has experience with something similar or are there some
> hints about how to optimize the postgres cluster for such bundled statements?
I think at this step you want to optimize for what the statements are doing,
not for the statements themselves. Could you send a query plan for the stored
procedure ?
Also, you'd maybe want to think if there's a way you can avoid making 100s of
1000s of requests per second, rather than trying to optimize for it. Can you
make another stored procedure which handles N requests rather than calling this
SP N times ? There's no guarantee that won't hit the same or other bottleneck,
until you see what that is.
--
Justin
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2020-09-08 10:49 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-09-08 10:30 Query Performance in bundled requests Dirk Krautschick <Dirk.Krautschick@trivadis.com>
2020-09-08 10:49 ` Justin Pryzby <pryzby@telsasoft.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox