public inbox for [email protected]
help / color / mirror / Atom feedFrom: Alena Rybakina <[email protected]>
To: Bertrand Drouvot <[email protected]>
To: Jim Nasby <[email protected]>
To: vignesh C <[email protected]>
Cc: Ilia Evdokimov <[email protected]>
Cc: pgsql-hackers <[email protected]>
Cc: Alexander Korotkov <[email protected]>
Cc: Kirill Reshke <[email protected]>
Cc: Andrei Zubkov <[email protected]>
Cc: Masahiko Sawada <[email protected]>
Cc: Melanie Plageman <[email protected]>
Cc: jian he <[email protected]>
Cc: [email protected]
Cc: Sami Imseih <[email protected]>
Subject: Re: Vacuum statistics
Date: Fri, 21 Mar 2025 22:46:41 +0300
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
<[email protected]>
<[email protected]>
<CAPpHfduoJEuoixPTTg2tjhnXqrdobuMaQGxriqxJ9TjN1uxOuA@mail.gmail.com>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<CAMFBP2oXkhX_k9FTqtW-LdTBepVq0PDuBEGO8-LpNGbyHTBrNw@mail.gmail.com>
<[email protected]>
<[email protected]>
Sorry, I forgot to provide a link to the problem [0], actually. So I
provided it below.
[0]
https://www.postgresql.org/message-id/CAPpHfduoJEuoixPTTg2tjhnXqrdobuMaQGxriqxJ9TjN1uxOuA%40mail.gma...
On 21.03.2025 22:42, Alena Rybakina wrote:
> On 13.03.2025 09:42, Bertrand Drouvot wrote:
>> Hi,
>>
>> On Wed, Mar 12, 2025 at 05:15:53PM -0500, Jim Nasby wrote:
>>> The usecase I can see here is that we don't want autovac creating so much
>>> WAL traffic that it starts forcing other backends to have to write WAL out.
>>> But tracking how many times autovac writes WAL buffers won't help with that
>> Right, because the one that increments the wal_buffers_full metric could "just"
>> be a victim (i.e the one that happens to trigger the WAL buffers disk flush,
>> even though other backends contributed most of the buffer usage).
>>
>>> (though we also don't want any WAL buffers written by autovac to be counted
>>> in the system-wide wal_buffers_full:
>> why? Or do you mean that it would be good to have 2 kinds of metrics: one
>> generated by "maintenance" activity and one by "regular" backends?
>>
>>> What would be helpful would be a way to determine if autovac was causing
>>> enough traffic to force other backends to write WAL. Offhand I'm not sure
>>> how practical that actually is though.
>> a051e71e28a could help to see how much WAL has by written by the autovac workers.
>>
>>> BTW, there's also an argument to be made that autovac should throttle
>>> itself if we're close to running out of available WAL buffers...
>> hmm, yeah I think that's an interesting idea OTOH that would mean to "delegate"
>> the WAL buffers flush to another backend.
>>
>> Regards,
>>
>
> I will add it and fix the tests but later and I'll explain why.
>
> I'm working on this issue [0] and try have already created new
> statistics in Statistics Collector to store database and relation
> vacuum statistics: PGSTAT_KIND_VACUUM_DB and PGSTAT_KIND_VACUUM_RELATION.
>
> Vacuum statistics are saved there instead of relation's and database's
> statistic structure, but for some reason it is not possible to find
> them in the hash table when building a snapshot and display them
> accordingly.
> I have not yet figured out where the error is.
>
> Without solving this problem, committing vacuum statistics is not yet
> possible. An alternative way for us was to refuse some statistics for
> now for relations,
> but we could not agree on which statistics should not be displayed yet
> and for now we are only adding them :).
>
> I understand why this is important to display more vacuum information
> about vacuum statistics - it will allow us to better understand the
> problems of incorrect vacuum settings or, for example, notice a bug in
> its operation.
>
> In order to reduce the memory consumption for storing them for those
> who are not going to use them, I just realized that we need to create
> a separate space for storing the statistics
> I mentioned above (PGSTAT_KIND_VACUUM_DB and
> PGSTAT_KIND_VACUUM_RELATION), there is no other way to do this and I
> am still trying to complete this functionality.
>
> I doubt that I will have time for this by code freeze date and even if
> I do, I will hardly have time for a normal review. There's really a
> lot more to learn related to the stat collector, so
> I'm postponing it to the next commitfest.
>
> Sorry. I'll fix the tests as soon as I finish this part, since they'll
> most likely either break the same way or in some new way.
>
> Tomorrow or the day after tomorrow I will send a diff patch with what
> I have already managed to demonstrate the problem, since I need to
> bring the code to a normal form.
> Maybe someone who worked with the stat collector will suddenly tell me
> where and what I have implemented incorrectly.
>
--
Regards,
Alena Rybakina
Postgres Professional
view thread (53+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Vacuum statistics
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox