Message-ID: <9af7cb4e-d191-421f-85a2-22d8c406b47a@tantorlabs.com>
Date: Thu, 9 Jan 2025 00:25:15 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Sample rate added to pg_stat_statements
From: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
To: Sami Imseih <samimseih@gmail.com>,
 "Andrey M. Borodin" <x4mmm@yandex-team.ru>
Cc: Alexander Korotkov <aekorotkov@gmail.com>,
 Michael Paquier <michael@paquier.xyz>,
 Greg Sabino Mullane <htamfids@gmail.com>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
References: <fe99e0ca-e564-480e-b865-5f0cee30bc60@tantorlabs.com>
 <c7ce658a-e0eb-4c5f-92a8-511314d89bbb@tantorlabs.com>
 <6707FCA9-FD1A-4609-A1A6-142456C14E0C@yandex-team.ru>
 <CAKAnmmLD8C-Y6CVF74Ccup8h9YpzBYzSvWWRG_Qf8+5YEMh4-g@mail.gmail.com>
 <Zz0MFPq1s4WFxxpL@paquier.xyz>
 <CAPpHfdsTKAQqC3A48-MGQhrhfEamXZPb64w=utk7thQcOMNr7Q@mail.gmail.com>
 <320a31e6-4c91-438b-ab17-8a1d72384727@tantorlabs.com>
 <f202ed0f-28c3-45aa-884c-db0a78322342@tantorlabs.com>
 <fb3df2e0-2cd4-49ea-bfda-148bc43d613d@tantorlabs.com>
 <52e5e3aa-066a-4daf-8d65-77c34a2f0d98@tantorlabs.com>
 <6482A576-4465-4E46-A42C-F1767E8DDE30@yandex-team.ru>
 <CAA5RZ0s=JH4owVr=Tfr4DYpFnBrYJjKoq2d7NqdkBQKY5whiRQ@mail.gmail.com>
 <20994d10-ab44-4c6b-91bc-c4675cd2e58e@tantorlabs.com>
 <9D8A5BC8-A3A5-4DA9-9382-F30D343247D1@yandex-team.ru>
 <CAA5RZ0sBsajHdE_HPoJ-C5rp8QmfArd6a6yJo6bDMVpyW25E+Q@mail.gmail.com>
 <6cf6270b-e8fa-4d99-b74a-17e781cdf309@tantorlabs.com>
Content-Language: en-US
In-Reply-To: <6cf6270b-e8fa-4d99-b74a-17e781cdf309@tantorlabs.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/9af7cb4e-d191-421f-85a2-22d8c406b47a%40tantorlabs.com>
Precedence: bulk


On 08.01.2025 22:39, Ilia Evdokimov wrote:
>
> After the changes in v9-patch, won’t all the necessary queries now be 
> normalized? Since there are no longer any restrictions in the parser 
> hook, queries will be normalized as usual, and pgss_planner, 
> pgss_ExecutorStart, and ExecutorEnd will simply fetch them from 
> 'pgss_query_texts.stat' file.
>
> For now, this seems like the simplest approach instead of making 
> JumbleState available to other hooks. Moreover, if we do proceed with 
> that, we might be able to remove the 'pgss_query_texts.stat' file 
> altogether and more other improvements. In my view, if no other 
> options arise, we should address making JumbleState available to other 
> hooks in a separate thread. If you have any suggestions, I'm always 
> open to feedback.
>
> I attached v9 patch with changes and test above.


Unfortunately, these changes do not achieve the intended sampling goal. 
I looked into this more deeply: while the sampled-out queries do not 
appear in pg_stat_statements, an entry is still allocated in the hash 
table after normalization, which, in my view, should not happen when 
sampling is in effect. Therefore, patch v9 is unlikely to meet our needs.

--
Best regards,
Ilia Evdokimov,
Tantor Labs LLC.