Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pXA2r-0002Sa-Fa for pgsql-hackers@arkaria.postgresql.org; Wed, 01 Mar 2023 00:10:09 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pXA2q-0008DS-1R for pgsql-hackers@arkaria.postgresql.org; Wed, 01 Mar 2023 00:10:08 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pXA2p-0008DF-K5 for pgsql-hackers@lists.postgresql.org; Wed, 01 Mar 2023 00:10:07 +0000 Received: from out4-smtp.messagingengine.com ([66.111.4.28]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pXA2l-0006LC-RI for pgsql-hackers@lists.postgresql.org; Wed, 01 Mar 2023 00:10:06 +0000 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 508935C010A; Tue, 28 Feb 2023 19:10:03 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Tue, 28 Feb 2023 19:10:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paquier.xyz; h= cc:cc:content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1677629403; x=1677715803; bh=K5qUe2os0T 41kZ+tLyieSVK30nBo1OcV4ahFirY3RFg=; b=f5sOIZOpxaSMuuNbFQQqiuMFgR DAV1Lgcu5DdaIx+l7AAyR8f5HqLB9L0m74H3ge9annMyKaW7OeXF1LzVu3V/axVz 4Tjpmn2uk61aJ6y/X+0dDHLItrQ8PG5oOL9O0t8Zq8yZuS4zXVUuRNTKrX1pl/Zb Cx6xNdi8XCc3oJ6DFTursuEpK2U7qVSGOibhj4u4U/XiypheDhEhJKzxXEBkh/VF m6DkMDfqIwjG5eKpQB+mI/LNu/KISEBlUxSCt/9sk50BOJoo3QdQ7vleDZsRAkQJ nQDMWWJa2m+rWMcHIPyVs96uuWg5TsnP94erg4DhTg6UDKVDPZ9Z7V4alRrg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1677629403; x=1677715803; bh=K5qUe2os0T41kZ+tLyieSVK30nBo 1OcV4ahFirY3RFg=; b=mYPQvk1/aIr2hwo9EE22Bg2fd3y6oYZsx4ou/kL7zrmE dFESEGedOB4aeWR4W1FyxTrB03sqOHSAsFnmhqLbZxQlPOskXKVFsbBK7a6gI/3s +hUmc8EE2OL8LuqV0lFce3TOyHLhRkZUZPKheDFuPMqlwCQLaL2+0+6EAkdC9WIW fElOoKhvb82EgqFVXKaUDzwUr3f668DlWqu7+PsTgPsiI8yVdQLXh8dix81xnIAw Wo7uc0sKeAlHnyN/kJUIqcJVYAL+p76QLwfGx/ndhu3GlbuQiMMNzZ/pXGG1T/Bq fLuM3Gcmv/Ik4VCemIZzMiUA/lKlms2DNp1s9ZJ/Uw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrudelgedgudegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne gfrhhlucfvnfffucdljedtmdenucfjughrpeffhffvvefukfhfgggtuggjsehgtderredt tddvnecuhfhrohhmpefoihgthhgrvghlucfrrghquhhivghruceomhhitghhrggvlhesph grqhhuihgvrhdrgiihiieqnecuggftrfgrthhtvghrnhepteelieefudffhffhtdetleeg geegfffhkeeuveetiefgudduvedutefggeeivdejnecuvehluhhsthgvrhfuihiivgeptd enucfrrghrrghmpehmrghilhhfrhhomhepmhhitghhrggvlhesphgrqhhuihgvrhdrgiih ii X-ME-Proxy: Feedback-ID: i0fe9450f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 Feb 2023 19:10:01 -0500 (EST) Date: Wed, 1 Mar 2023 09:09:57 +0900 From: Michael Paquier To: "Imseih (AWS), Sami" Cc: PostgreSQL Hackers Subject: Re: Doc update for pg_stat_statements normalization Message-ID: References: <9CFF3512-355B-4676-8CCC-6CF622F4DC1A@amazon.com> <8042E3F8-0BC8-4194-9F4C-32071389552A@amazon.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="EGhBHrxkwPYXz5xM" Content-Disposition: inline In-Reply-To: <8042E3F8-0BC8-4194-9F4C-32071389552A@amazon.com> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --EGhBHrxkwPYXz5xM Content-Type: multipart/mixed; boundary="FRtDZb5Zt4mPHzkc" Content-Disposition: inline --FRtDZb5Zt4mPHzkc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Feb 28, 2023 at 11:11:30PM +0000, Imseih (AWS), Sami wrote: > I agree. We introduce the concept of a plannable statement in a=20 > previous section and we can then make this distinction in the new > paragraph.=20 >=20 > I also added a link to pg_stat_statements_info since that is introduced > later on int the doc. I have reworded the paragraph a bit to be more general so as it would not need an update once more normalization is applied to utility queries (I am going to fix the part where we mention that we use the strings for utilities, which is not the case anymore now): + + Queries on which normalization can be applied may be observed with cons= tant + values in pg_stat_statements, especially when = there + is a high rate of entry deallocations. To reduce the likelihood of this + happening, consider increasing pg_stat_statements.max. + The pg_stat_statements_info view, discussed be= low + in , + provides statistics about entry deallocations. + Are you OK with this text? -- Michael --FRtDZb5Zt4mPHzkc Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0001-doc-update-regarding-pg_stat_statements-normaliza.patch" Content-Transfer-Encoding: quoted-printable =46rom dd8938e4ba1b1f29d14b3fa2dc76301f42592cad Mon Sep 17 00:00:00 2001 =46rom: Michael Paquier Date: Wed, 1 Mar 2023 09:05:08 +0900 Subject: [PATCH v3] doc update regarding pg_stat_statements normalization. It is quite possible that a query text willl not normalize (replace constants with $1, $2..) when there is a high rate pgss_hash deallocation. This commit calls this out in docs. --- doc/src/sgml/pgstatstatements.sgml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/doc/src/sgml/pgstatstatements.sgml b/doc/src/sgml/pgstatstatem= ents.sgml index efc36da602..f1ba78c8cb 100644 --- a/doc/src/sgml/pgstatstatements.sgml +++ b/doc/src/sgml/pgstatstatements.sgml @@ -516,6 +516,16 @@ pg_stat_statements entry. =20 + + Queries on which normalization can be applied may be observed with cons= tant + values in pg_stat_statements, especially when = there + is a high rate of entry deallocations. To reduce the likelihood of this + happening, consider increasing pg_stat_statements.max. + The pg_stat_statements_info view, discussed be= low + in , + provides statistics about entry deallocations. + + In some cases, queries with visibly different texts might get merged in= to a single pg_stat_statements entry. Normally thi= s will happen --=20 2.39.2 --FRtDZb5Zt4mPHzkc-- --EGhBHrxkwPYXz5xM Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmP+l9UACgkQnvQgOdby QH3zYg//YAaYXwVW/Mc9KAbyZpoDqU9e8r1CBQkoEa5fwYi7vOcK9Ko7rE4i7jRM i8aUq/Cez0WHQZbSUjJD/ce+OHLUcJDGRfJicoLktSjyZOjc2svzrMn4Srj8mEnS ngaEuLYzb3KUPmMbASwbOloN3lLJJthjck1HCeNp0PYahAXT7ZEFuIbYMJYKEHWc Jx1gjDpLopc4J9zZZP780AfPf8LJU4Ga4lPqIz/8KCYrgSjyaVIqv1OIerYoR71F JpMYzqlFLm/X5e860cdh7SuoIgrozUUj22ZpuDs1eXBnWPsUlJUwIEcNu4a7OHOB j+ev+YxMQFUQPDFgHU/Ti0scj2BP1Lz/kvp7Cl8tKh3Qn2qoj7d7QtXsc0ThgoRj bPCn+pUrO7IFbvIQR2D7tLOh5zuStqmxBKvoTyAUVol3vs6mD9uN9iWycDfrpLB7 yxvB32hK6SQt0uB3mYAA3hGmWBDVds7kSr8jDW4SdiJzVxr1s/ukQhxy5b5+QC/G bNfCQO98w7N0FR3q9AvC36uS7aHCDizk3KHfKYB9tcCfPhnDvGToeUaxcF9PiOzV a5pw4TohYq6DZpc0ddz0fnSBoS/e3IYYYa5Ol7XSFrP7HH+qWY6k8e5ngY/JpoYN fE1q+tHQxjOAXxoVjhzqeAb1oyJmxbRZQFck5WcsctQ2qmFp03E= =rvYS -----END PGP SIGNATURE----- --EGhBHrxkwPYXz5xM--