Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\))
Subject: Re: Sample rate added to pg_stat_statements
From: "Andrey M. Borodin" <x4mmm@yandex-team.ru>
In-Reply-To: <c7ce658a-e0eb-4c5f-92a8-511314d89bbb@tantorlabs.com>
Date: Tue, 19 Nov 2024 17:11:55 +0500
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <6707FCA9-FD1A-4609-A1A6-142456C14E0C@yandex-team.ru>
References: <fe99e0ca-e564-480e-b865-5f0cee30bc60@tantorlabs.com>
 <c7ce658a-e0eb-4c5f-92a8-511314d89bbb@tantorlabs.com>
To: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Archived-At: <https://www.postgresql.org/message-id/6707FCA9-FD1A-4609-A1A6-142456C14E0C%40yandex-team.ru>
Precedence: bulk


> On 18 Nov 2024, at 23:33, Ilia Evdokimov =
<ilya.evdokimov@tantorlabs.com> wrote:
>=20
> Hi hackers,
>=20
> Under high-load scenarios with a significant number of transactions =
per second, pg_stat_statements introduces substantial overhead due to =
the collection and storage of statistics. Currently, we are sometimes =
forced to disable pg_stat_statements or adjust the size of the =
statistics using pg_stat_statements.max, which is not always optimal. =
One potential solution to this issue could be query sampling in =
pg_stat_statements.
>=20
> A similar approach has been implemented in extensions like =
auto_explain and pg_store_plans, and it has proven very useful in =
high-load systems. However, this approach has its trade-offs, as it =
sacrifices statistical accuracy for improved performance. This patch =
introduces a new configuration parameter, pg_stat_statements.sample_rate =
for the pg_stat_statements extension. The patch provides the ability to =
control the sampling of query statistics in pg_stat_statements.
>=20
> This patch serves as a proof of concept (POC), and I would like to =
hear your thoughts on whether such an approach is viable and applicable.

+1 for the idea. I heard a lot of complaints about that pgss is costly. =
Most of them were using it wrong though. But at least it could give an =
easy way to rule out performance impact of pgss.

> On 19 Nov 2024, at 15:09, Ilia Evdokimov =
<ilya.evdokimov@tantorlabs.com> wrote:
>=20
> I believe we should also include this check in the pgss_ExecutorEnd() =
function because sampling in pgss_ExecutorEnd() ensures that a query not =
initially sampled in pgss_ExecutorStart() can still be logged if it =
meets the pg_stat_statements.sample_rate criteria. This approach adds =
flexibility by allowing critical queries to be captured while =
maintaining efficient sampling.

Is there a reason why pgss_ProcessUtility is excluded?


Best regards, Andrey Borodin.=