public inbox for [email protected]  

help / color / mirror / Atom feed

From: Alena Rybakina <[email protected]>
To: Amit Kapila <[email protected]>
Cc: Alexander Korotkov <[email protected]>
Cc: pgsql-hackers <[email protected]>
Cc: Jim Nasby <[email protected]>
Cc: Bertrand Drouvot <[email protected]>
Cc: Ilia Evdokimov <[email protected]>
Cc: Kirill Reshke <[email protected]>
Cc: Andrei Zubkov <[email protected]>
Cc: Masahiko Sawada <[email protected]>
Cc: Melanie Plageman <[email protected]>
Cc: jian he <[email protected]>
Cc: [email protected]
Cc: Sami Imseih <[email protected]>
Cc: vignesh C <[email protected]>
Subject: Re: Vacuum statistics
Date: Tue, 13 May 2025 05:49:02 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAA4eK1JOUn+EqWSfRgKfgBZOXT7Q2dw2enmSZZgOhoMOFwopPA@mail.gmail.com>
References: <[email protected]>
	<[email protected]>
	<[email protected]>
	<CAPpHfduoJEuoixPTTg2tjhnXqrdobuMaQGxriqxJ9TjN1uxOuA@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<CAMFBP2oXkhX_k9FTqtW-LdTBepVq0PDuBEGO8-LpNGbyHTBrNw@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<CAA4eK1JOUn+EqWSfRgKfgBZOXT7Q2dw2enmSZZgOhoMOFwopPA@mail.gmail.com>

Hi!

On 12.05.2025 08:30, Amit Kapila wrote:
> On Fri, May 9, 2025 at 5:34 PM Alena Rybakina <[email protected]> wrote:
>> I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and
>> this allows us to solve the problem with the memory consumed.
>>
> I think this patch is trying to collect data similar to what we do for
> pg_stat_statements for SQL statements. So, can't we follow a similar
> idea such that these additional statistics will be collected once some
> external module like pg_stat_statements is enabled? That module should
> be responsible for accumulating and resetting the data, so we won't
> have this memory consumption issue.
The idea is good, it will require one hook for the pgstat_report_vacuum 
function, the extvac_stats_start and extvac_stats_end functions can be 
run if the extension is loaded, so as not to add more hooks.
But I see a problem here with tracking deleted objects for which 
statistics are no longer needed. There are two solutions to this and I 
don't like both of them, to be honest.
The first way is to add a background process that will go through the 
table with saved statistics and check whether the relation or the 
database are relevant now or not and if not, then
delete the vacuum statistics information for it. This may be 
resource-intensive. The second way is to add hooks for deleting the 
database and relationships (functions dropdb, index_drop, 
heap_drop_with_catalog).
> BTW, how will these new statistics be used to autotune a vacuum?
yes, but they are collected on demand - by guc.
> And
> do we need all the statistics proposed by this patch?
>
Regarding this issue, it was discussed here and so far we have come to 
the conclusion that statistics are needed for a deep understanding of 
the work of vacuum statistics [0] [1] [2].

[0] 
https://www.postgresql.org/message-id/0B6CBF4C-CC2A-4200-9126-CE3A390D938B%40upgrade.com

[1] 
https://www.postgresql.org/message-id/6732acf8ce0f31025b535ae1a64568750924a887.camel%40moonset.ru

[2] 
https://www.postgresql.org/message-id/5AA8FFD5-6DE2-4A31-8E00-AE98F738F5D1%40upgrade.com


-- 
Regards,
Alena Rybakina
Postgres Professional

view thread (53+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: Vacuum statistics
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox