public inbox for [email protected]
help / color / mirror / Atom feedFrom: Daniil Davydov <[email protected]>
To: Masahiko Sawada <[email protected]>
Cc: Sami Imseih <[email protected]>
Cc: Alexander Korotkov <[email protected]>
Cc: Matheus Alcantara <[email protected]>
Cc: Maxim Orlov <[email protected]>
Cc: Postgres hackers <[email protected]>
Subject: Re: POC: Parallel processing of indexes in autovacuum
Date: Tue, 10 Feb 2026 22:03:45 +0700
Message-ID: <CAJDiXggY1QzNde6_HhpzneLc9dYqmWZ+PY39cuBXYdcCTuoJBA@mail.gmail.com> (raw)
In-Reply-To: <CAD21AoD7_4gsQ2a82zO3SaRwjdw_3tyiYDHNFPUKQ5DAA5HOtA@mail.gmail.com>
References: <CACG=ezZOrNsuLoETLD1gAswZMuH2nGGq7Ogcc0QOE5hhWaw=cw@mail.gmail.com>
<CAD21AoCdx5ZNS_cO7bYz1Zfb+Kw1kuJV2wtewrz7T1pPpjcWGw@mail.gmail.com>
<CAJDiXgi6ZQOoSEqj9RyZMEh+HHBtmW0+PHD85UNPtKch8ubvdg@mail.gmail.com>
<CAD21AoBcoA-i-pJ_=y+jg14R8_QaJA1iwktCnu5i-C=yXDFPdA@mail.gmail.com>
<CAJDiXgjnUdE6Sk4M0unmT+9dULyFAxcum2txQKpWTuo4uQ_oXQ@mail.gmail.com>
<CAD21AoBTZdVR93JBo620B=MX-K8cdm3VRbjrBr_Vcpngk3AjVw@mail.gmail.com>
<CAA5RZ0vfBg=c_0Sa1Tpxv8tueeBk8C5qTf9TrxKBbXUqPc99Ag@mail.gmail.com>
<CAD21AoBgvUeWS8ZsXBahA1XdYayK6DJ6dx49d6Xpii-iH+Hrwg@mail.gmail.com>
<CAA5RZ0vF+Lr-jU1LAZWTGUjboUETk8oLvaNBbA5ozX6dau+how@mail.gmail.com>
<CAJDiXggueLSGMNRmLshbmFRfbo4jzks0W8bLDfUSRZ-61fPVEQ@mail.gmail.com>
<CAFY6G8cJ=DRTX75pOGerH6sk39dRt+7MSH+y_qppDdhPs=qdQA@mail.gmail.com>
<CAJDiXgg1t6wk9NjyMUTm1iKqM9GtdQ_wrEchBtz3xjWBZM8W8A@mail.gmail.com>
<CAD21AoAC0=Xi38RQcAO4A+vdmoXToZMoHfbS=KLT49fAOTH_gA@mail.gmail.com>
<CAJDiXgiD+AZKhJSn-FSRVQxtDLmJd95wDu4wtKniQF5==1JcjQ@mail.gmail.com>
<CAD21AoAM8KsqNhrZYJuf7odvxcTC0TumXazJc-r_wC5KnDFDPg@mail.gmail.com>
<CAJDiXghbcOC9OOj3ampxuyqXH0geggnosnrYUHGygkpss-RtxA@mail.gmail.com>
<CAD21AoAPnq0vrcGgeN++r1GoL8Kza7jaGL=TNzuBn6+MkR=rUQ@mail.gmail.com>
<CAJDiXghmsbTmnm--9B5bbuZXa1OL7SZ0HYppX3tx9XsdwfJBhA@mail.gmail.com>
<[email protected]>
<CAJDiXgiYiX+azuR76DcVx8fZn57m_4v6cB14-GW34mWa=qudFQ@mail.gmail.com>
<CAD21AoDtPpkkQ_h1yf4oTx1qn4SRdTeVY3qs+9J07fYqa_4Gww@mail.gmail.com>
<CAJDiXgi7KB7wSQ=Ux=ngdaCvJnJ5x-ehvTyiuZez+5uKHtV6iQ@mail.gmail.com>
<CAD21AoCcHKKXsr9Oh736ejckqqS1i430xGEyJ=JP5OL0ExyP1A@mail.gmail.com>
<CAJDiXghaFT_1sSv3q8mjyZ_RLZDgiogg0mWRvLxSWvkUi2CcLg@mail.gmail.com>
<CAA5RZ0u63W41OmcEO+HLs4CSo-Sd3J+Q-4=04iud8V=xX4iUrA@mail.gmail.com>
<CAJDiXgin1TXniVGJKzOTA=F9K342uVfm6O0EmubTVB=F+XSrbA@mail.gmail.com>
<CAD21AoDadzAwibxf-+urjx=XL+eVu8=Ut-Lh2GxXUt32LbPG3Q@mail.gmail.com>
<CAD21AoD6HhraqhOgkQJOrr0ixZkAZuqJRpzGv-B+_-ad6d5aPw@mail.gmail.com>
<CAJDiXgiGSpqMQSOx-cVO_LtcB5GWHBy9ph7oOR4ebbX8A==kgw@mail.gmail.com>
<CAD21AoBRRXbNJEvCjS-0XZgCEeRBzQPKmrSDjJ3wZ8TN28vaCQ@mail.gmail.com>
<CAPpHfduBJfMcojvmYHUo8b_C=0cxRy1N+tNiNGoA3RAZq2ApaA@mail.gmail.com>
<CAD21AoC82NeHKXc965pPUZO2eyo1U7P6cmfRJbrcPDcnd7_6hw@mail.gmail.com>
<CAJDiXghP2kXnEz+cj3rAWNM3NdKSB_4WtnngFXpVz2omPhGr5A@mail.gmail.com>
<CAD21AoA0bnRZC_OqKMnH-Ln+OZ9z9k56j2c_MXj8pw69O-wkBw@mail.gmail.com>
<CAA5RZ0sSXDza7_nUUbhHL_Sws+M+HR1daKJPXHpdLuNCkwUgUg@mail.gmail.com>
<CAJDiXggrBsbzOisf+Nu8pZkYGrpUZaFbosL1Wbm3kKxzTm4xgw@mail.gmail.com>
<CAA5RZ0tbiPcgQEjnhdnjz6qSjfRsGrr8jGCaMcrMaoPpax3wig@mail.gmail.com>
<CAJDiXgjt5ZmK2uvS0E8Ztt5ePYmq8Ze_dG05Zo2NUsKLHCEuYA@mail.gmail.com>
<CAD21AoB7v5tLPXLK=qmtt6PaEC1f+Fb-gh+MwAbXfm6x4eZGNw@mail.gmail.com>
<CAJDiXghwtUbiFnAh3nSaxTk8KFupQuMbp+g4z3wOLoQfMuqgDg@mail.gmail.com>
<CAJDiXgjoNd4BF19HNY_FAcDUqiqsfw8cGhNOJwBxahB8P38E3Q@mail.gmail.com>
<CAD21AoBT1LWqPZkcHpVMVh0ZOXUneO=p61t0i8cQ+kOP9qfODQ@mail.gmail.com>
<CAJDiXggL=J0nV7PfBsMW9+UOU3KUp1jNBM9Gov1JvAX7aG_U1g@mail.gmail.com>
<CAD21AoDz-1Zf9DOJJrdcB2=eNA4UdywthkowNp_dHmOGC-yV_g@mail.gmail.com>
<CAJDiXgjzphJ313=aDwbvryHpmTi6AqE+-5crysTtzKv01-vkzA@mail.gmail.com>
<CAD21AoD7_4gsQ2a82zO3SaRwjdw_3tyiYDHNFPUKQ5DAA5HOtA@mail.gmail.com>
Hi,
Thanks everyone for the review!
**Comments on the 0001 patch**
On Thu, Jan 22, 2026 at 5:22 AM Sami Imseih <[email protected]> wrote:
>
>
> + AutoVacuumShmem->av_freeParallelWorkers = Max(nfree_workers, 0);
> +
> + /*
> + * Don't allow number of free workers to become less than zero if the
> + * patameter was decreased.
> + */
> + AutoVacuumShmem->av_freeParallelWorkers =
> + Max(AutoVacuumShmem->av_freeParallelWorkers, 0);
> ```
>
> This can probably be simplified to:
>
> ```
> AutoVacuumShmem->av_freeParallelWorkers = Max(nfree_workers, 0);
> ```
On Thu, Jan 22, 2026 at 5:29 AM Masahiko Sawada <[email protected]> wrote:
>
> + /*
> + * Cap or increase number of free parallel workers according to the
> + * parameter change.
> + */
> + AutoVacuumShmem->av_freeParallelWorkers = Max(nfree_workers, 0);
> +
> + /*
> + * Don't allow number of free workers to become less than zero if the
> + * patameter was decreased.
> + */
> + AutoVacuumShmem->av_freeParallelWorkers =
> + Max(AutoVacuumShmem->av_freeParallelWorkers, 0);
>
> Why does it do Max(x, 0) twice?
Agreed, I missed this one. Surely it can be simplified.
--
On Thu, Jan 22, 2026 at 5:29 AM Masahiko Sawada <[email protected]> wrote:
>
> On Sat, Jan 17, 2026 at 6:52 AM Daniil Davydov <[email protected]> wrote:
> >
> > I will keep the 'max_worker_processes' limit, so autovacuum will not
> > waste time initializing a parallel context if there is no chance that
> > the request will succeed.
> > But it's worth remembering that actually the
> > 'autovacuum_max_parallel_workers' parameter will always be implicitly
> > capped by 'max_parallel_workers'.
>
> It doesn't make sense to me that we limit
> autovacuum_max_parallel_workers by max_worker_processes TBH. When
> users want to have more parallel vacuum workers for autovacuum and the
> VACUUM command, they would have to consider max_worker_processes,
> max_parallel_workers, and max_parallel_maintenance_workers separately.
> Given that max_parallel_workers is controlling the number of
> max_worker_processes that can be used in parallel operations, I
> believe that parallel vacuum workers for autovacuum should also be
> taken from that pool.
Maybe I don't quite understand the meaning of "limited by". For example,
we have a max_parallel_workers_per_gather parameter, which is limited
by max_parallel_workers. But actually we can set this parameter to a value
that is higher than max_parallel_workers. The limitation is that for Gather
node we cannot request more workers than are available in bgworkers pool
(where number of free workers is always <= max_parallel_workers). Thus,
limitation actually exists only for bgworkers pool, on which other parallel
operations depend. In particular, whatever values we set for the
autovacuum_max_parallel_workers parameter, it always will depend only
on bgworkers pool.
I'll give in to your opinion and add a limitation by max_parallel_workers.
But I still don't understand where the point is in explicit limitation by
max_parallel_workers, if we already have this limitation implicitly?
It seems a bit redundant for me. I hope I've conveyed my point correctly.
**Comments on the 0002 patch**
On Thu, Jan 22, 2026 at 5:22 AM Sami Imseih <[email protected]> wrote:
>
> I don't think showing "reserved" in the logging is needed and could be
> confusing.
>
The rationale for this is in the previous letter of Masahiko-san, and I
agree with it. Do you think it can be confusing because users
aren't familiar with the "reserved workers" in terms of postgres?
I think that we can write about it in documentation, so users will
be ready for it.
--
On Thu, Jan 22, 2026 at 5:22 AM Sami Imseih <[email protected]> wrote:
>
> I think it's better
> to show separate lines for index vacuuming and index cleanup, just
> like VACUUM VERBOSE.
>
> ```
> INFO: launched 2 parallel vacuum workers for index vacuuming (planned: 2)
> INFO: launched 1 parallel vacuum worker for index cleanup (planned: 1)
> ```
>
Actually, we already have such a logging (see
parallel_vacuum_process_all_indexes function) for both VACUUM
PARALLEL and parallel autovacuum. I think that in addition we can
split the final log message (with total parallel vacuum stats) into two
lines : for vacuum and cleanup respectively. Please, see these changes
in the 0002 patch.
--
On Thu, Jan 22, 2026 at 5:29 AM Masahiko Sawada <[email protected]> wrote:
>
> Can we refactoring these codes to:
>
> if (vacrel->workers_usage.nplanned > 0)вв
> {
> if (AmAutoVacuumWorkerProcess())
> appendStringInfo(...);
> else
> appendStringInfo(...);
I agree.
**Comments on the 0003 patch**
On Thu, Jan 22, 2026 at 5:29 AM Masahiko Sawada <[email protected]> wrote:
>
> On Sat, Jan 17, 2026 at 6:52 AM Daniil Davydov <[email protected]> wrote:
> >
> > If behavior will be changed, then all comments for this function will need to
> > be changed, actually. Don't get me wrong - I just think that this Note is
> > important for the readers. But if you doubt its usefulness, I don't
> > mind deleting it.
>
> I still could not figure out why it should be mentioned here instead
> of at the comment of parallel_vacuum_update_shared_delay_params().
> Readers can notice that calling
> parallel_vacuum_update_shared_delay_params() for parallel vacuum
> worker for the VACUUM command has no effect when reading the function.
> In my opinion, we should mention here why we call
> parallel_vacuum_update_shared_delay_params() but should not mention
> what the called function does because it should have been described in
> that function.
>
OK, I agree.
> BTW can we expose pv_shared_cost_params so that we can check it in
> vacuum_delay_point() before trying to call
> parallel_vacuum_update_shared_delay_params()?
>
I would prefer not to do so. IMO it would be better if we'll encapsulate the
shared delay parameters logic inside a single file.
> + if (!AmAutoVacuumWorkerProcess() && IsParallelWorker())
> + {
>
> We can just check IsParallelWorker() here.
I agree.
--
On Thu, Jan 22, 2026 at 5:29 AM Masahiko Sawada <[email protected]> wrote:
>
> +extern void parallel_vacuum_update_shared_delay_params(void);
> +extern void parallel_vacuum_propagate_cost_based_params(void);
>
> I think it's better to have similar names to these functions for
> consistency and readability. How about the following?
>
> parallel_vacuum_update_delay_params();
> parallel_vacuum_propagate_delay_params();
>
Yep, 100% agree - I just forgot to do it. if you don't mind, I would leave
the word "shared" in the function names.
--
On Thu, Jan 22, 2026 at 5:29 AM Masahiko Sawada <[email protected]> wrote:
>
> + params_generation = pg_atomic_read_u32(&pv_shared_cost_params->generation);
> +
> + SpinLockAcquire(&pv_shared_cost_params->spinlock);
> +
> + pv_shared_cost_params->cost_delay = vacuum_cost_delay;
> + pv_shared_cost_params->cost_limit = vacuum_cost_limit;
> + pv_shared_cost_params->cost_page_dirty = VacuumCostPageDirty;
> + pv_shared_cost_params->cost_page_hit = VacuumCostPageHit;
> + pv_shared_cost_params->cost_page_miss = VacuumCostPageMiss;
>
> I think we can check if the new cost-based delay parameters are really
> changed before changing the shared values. If users don't change
> cost-based delay parameters, we don't need to increment the generation
> at all.
>
I agree.
--
On Thu, Jan 22, 2026 at 5:29 AM Masahiko Sawada <[email protected]> wrote:
>
> +/*
> + * Only autovacuum leader can reload config file. We use this structure in
> + * parallel autovacuum for keeping worker's parameters in sync with leader's
> + * parameters.
> + */
> +typedef struct PVSharedCostParams
>
> I'd suggest writing the overall description first (e.g., what the
> struct holds and what the function does etc), and then describing the
> details and notes etc. For instance, readers might be confused when
> reading the first sentence "Only autovacuum leader can reload config
> file" as the struct definition is not related to the autovacuum
> implementation fact that autovacuum workers can reload the config file
> during the work. We would need to mention such detail somewhere in the
> comments but I think it should not be the first sentence. How about
> rewriting it to something like:
>
> +/*
> + * Struct for cost-based vacuum delay related parameters to share among an
> + * autovacuum worker and its parallel vacuum workers.
> + */
>
Yep, you are right.
> + slock_t spinlock; /* protects all fields below */
>
> It's convention to name 'mutex' as a field name.
>
OK.
--
> +static PVSharedCostParams * pv_shared_cost_params = NULL;
> +
> +/* See comments for structure above for the explanation. */
> +static uint32 shared_params_generation_local = 0;
>
> I think it's preferable to move these definitions of static variables
> right before the function prototypes.
>
I agree.
--
> + /*
> + * If 'true' then we are running parallel autovacuum. Otherwise, we are
> + * running parallel maintenence VACUUM.
> + */
> + bool am_parallel_autovacuum;
>
> How about renaming it to use_shared_delay_params? I think it conveys
> better what the field is used for.
I think that we should leave this name, because in the future some other
behavior differences may occur between manual VACUUM and autovacuum.
If so, we will already have an "am_autovacuum" field which we can use in
the code.
The existing logic with the "am_autovacuum" name is also LGTM - we should
use shared delay params only because we are running parallel autovacuum.
--
On Thu, Jan 22, 2026 at 5:22 AM Sami Imseih <[email protected]> wrote:
>
> inside vacuum_delay_point, I would re-organize the checks to
> first run the code block for the a/v worker:
>
> ```
> if (ConfigReloadPending && AmAutoVacuumWorkerProcess())
> ```
>
> then the a/v/ parallel worker:
>
> ```
> if (!AmAutoVacuumWorkerProcess() && IsParallelWorker())
> ```
>
Besides ConfigReloadPending we also must check VacuumCostActive.
I placed the call of update_shared_delay_params function *before* checking
VacuumCostActive, because parallel worker can change value of this variable
inside of this function. Also we should call functions related to a/v worker
only *after* checking the VacuumCostActive. Thus, the parallel a/v worker
logic should be called before leader a/v worker logic.
Am I missing something?
--
On Thu, Jan 22, 2026 at 5:22 AM Sami Imseih <[email protected]> wrote:
>
> But I am also wondering if we should have a specific backend_type
> for "autovacuum parallel worker" to differentiate that from the
> existing "autovacuum worker".
>
> and also we can have a helper macro like:
> ```
> #define AmAutoVacuumParallelWorkerProcess() (MyBackendType ==
> B_AUTOVAC_PARALLEL_WORKER)
> ```
>
> What do you think?
>
I don't think that we should do it, because the workers (that are launched
by a/v worker) are technically no different from other bgworkers, that are
launched for other purposes. Since we easily can distinguish a/v parallel
worker from others, I suggest we leave it as it is.
--
On Thu, Jan 22, 2026 at 5:22 AM Sami Imseih <[email protected]> wrote:
>
> Add
> ```
> +typedef struct PVSharedCostParams
> ````
>
> to src/tools/pgindent/typedefs.list
>
I agree. I'll also add all new structures to the typedefs.list
--
On Thu, Jan 22, 2026 at 5:22 AM Sami Imseih <[email protected]> wrote:
>
>
> + pg_atomic_init_u32(&shared->cost_params.generation, 0);
> + SpinLockInit(&shared->cost_params.spinlock);
> + pv_shared_cost_params = &(shared->cost_params);
>
> NIT: move SpinLockInit last
I think that we should init the pointer to the shared->cost_params when
all of this structure's fields are initialized. Thus, I guess that SpinLockInit
should be placed before the "pv_shared_cost_params = ...".
Here it doesn't actually make any difference where to place it, but I think
It's a little more beautiful.
--
On Thu, Jan 22, 2026 at 5:22 AM Sami Imseih <[email protected]> wrote:
>
>
> Instead of:
>
> ```
> + params_generation =
> pg_atomic_read_u32(&pv_shared_cost_params->generation);
> +
> ```
> and then later on:
> ````
> + /*
> + * Increase generation of the parameters, i.e. let parallel workers know
> + * that they should re-read shared cost params.
> + */
> + pg_atomic_write_u32(&pv_shared_cost_params->generation,
> + params_generation + 1);
> +
> + SpinLockRelease(&pv_shared_cost_params->spinlock);
> ```
>
> why can't we just do:
>
> pg_atomic_fetch_add_u32(&pv_shared_cost_params->generation, 1);
>
On Thu, Jan 22, 2026 at 5:29 AM Masahiko Sawada <[email protected]> wrote:
>
> + pg_atomic_write_u32(&pv_shared_cost_params->generation,
> + params_generation + 1);
>
> We can use pg_atomic_add_fetch_u32() instead.
>
Yep, agreed.
--
On Thu, Jan 22, 2026 at 5:22 AM Sami Imseih <[email protected]> wrote:
>
> Also, do the pg_atomic_fetch_add_u32 outside of the spinlock. right?
>
Sure. Somehow I missed it.
**Comments on the 0004 patch**
On Thu, Jan 22, 2026 at 5:29 AM Masahiko Sawada <[email protected]> wrote:
>
> The patch introduces 5 injection points, which seems overkill to me
> for implementing the tests. IIUC we can implement the test2 with two
> injection points: 'autovacuum-start-parallel-vacuum' (set right before
> lazy_scan_heap() call) and
> 'autovacuum-leader-before-indexes-processing'.
>
> 1. stop the autovacuum worker at 'autovacuum-start-parallel-vacuum'.
> 2. change delay params and reload the conf.
> 3. let the autovacuum worker process tables (vacuum_delay_point() is
> called during the heap scan).
> 4. stop the autovacuum worker at 'autovacuum-leader-before-indexes-processing'.
> 5. let parallel workers process indexes (vacuum_delay_point() is
> called during index vacuuming).
OK, I'll do it.
--
> For test3, I think we can write a DEBUG2 log in
> adjust_free_parallel_workers() and check it during the test instead of
> introducing the test-only function.
>
> Truncating all logs every after test would decrease the debuggability
> much. We can pass the offset as the start point to wait for the
> contents.
>
I've combined two of your above comments purposely. I agree that truncating
all logs is a bad decision and we need to solve this in a different way. But the
problem will occur If we want to 1) use logging instead of a test-only function
and 2) use offsets as the start point to wait for the contents in the logfile.
Imagine that we (using the described approach) need to wait until the end of
parallel index processing and determine the current number of free parallel
workers.
IIUC, It'll look like this :
wait_for_av_log("autovacuum processing finished");
wait_for_av_log("number of free workers = N");
But when we call wait_for_av_log first time, we will advance "offset" to the
end of logfile and thus we will miss the "number of free workers = N". The
only way to avoid it is to write a function that will determine the exact
position of "autovacuum processing finished" in the logfile. Isn't it too much?
I think that using wait_for_av_log("autovacuum processing finished"); +
SELECT get_parallel_autovacuum_free_workers(); will be much more
demonstrably and simply.
Moreover, the AutoVacuumGetFreeParallelWorkers function doesn't
seem completely useless in isolation from tests. I suggest leaving
this function and its usage in the tests. I can remove the "For testing
purpose only!" comment, so everyone will be free to use this function
in the future.
> For test4 and test5, we check the number of free workers using
> get_parallel_av_free_workers(). However, since autovacuum
> could retry to vacuum the table again, the test could fail.
Yep, good catch.
1)
Test 5 can be stabilized as follows :
We can attach to the "autovacuum-start-parallel-vacuum" injection point in
the "wait" mode. Thereby when we terminate the first a/v leader, we are
guaranteed that no other a/v leader will reach release/reserve functions.
And then we are free to call the get_parallel_autovacuum_free_workers
function. I'll additionally describe this logic in the test.
2)
In the test 4 I found another problem : when a/v leader errors out, it will
exit() pretty soon. And during exit() it will call the before_shmem_exit hook.
Thus, we cannot be sure that parallel workers has been released exactly
in the try/catch block. In order to guarantee it, I think that we should log
something inside the try/catch block. I added a pretty controversial loggin
code for it, but it is the best I came up with.
In the test 4 the above idea will look something like this:
$log_start = $node->wait_for_log(
qr/error triggered for injection point / .
qr/autovacuum-leader-before-indexes-processing/,
$log_start
);
$log_start = $node->wait_for_log(
qr/2 parallel autovacuum workers has been released after occured error/,
$log_start
);
Above I described a problem that may occur when we advance
"logfile offset" too far after the first wait_for_log call. Here, this problem
doesn't occur because the autovacuum launcher infinitely tries to
vacuum the table, so other "N workers released" messages occur.
--
> And here are some general comments and suggestions:
>
> +use warnings FATAL => 'all';
> +use PostgreSQL::Test::Cluster;
> +use PostgreSQL::Test::Utils;
> +use Test::More;
>
> We need comments to explain what we test with this test file.
>
OK, I'll add it. I suppose I can limit myself to a simple
"Test parallel autovacuum behavior", because the specific test scenarios
are described below.
--
> + $node->safe_psql('postgres', qq{
> + UPDATE test_autovac SET col_1 = $test_number;
> + ANALYZE test_autovac;
> + });
>
> Why do we need to execute ANALYZE as well?
I added ANALYZE just in case. But I see that statistics of deleted and
updated tuples is accumulated at the end of the transaction, so I agree
that we can get rid of ANALYZE here.
--
> +# Insert specified tuples num into the table
> +$node->safe_psql('postgres', qq{
> + DO \$\$
> + DECLARE
> + i INTEGER;
> + BEGIN
> + FOR i IN 1..$initial_rows_num LOOP
> + INSERT INTO test_autovac VALUES (i, i + 1, i + 2, i + 3);
> + END LOOP;
> + END \$\$;
> +});
>
> We can use generate_series() here. And it's faster to load the data
> and then create indexes.
OK, I'll fix it.
--
> +$node->psql('postgres',
> + "SELECT get_parallel_autovacuum_free_workers();",
> + stdout => \$psql_out,
> +);
>
> Please use pgsql_safe() instead.
Sure!
--
Again, thanks everyone for the review!
I hope I didn't miss anything.
Please, see updated sets of patches.
This time I'll try something experimental - besides the patches I'll also
post differences between corresponding patches from v20 and v21.
I.e. you can apply v20--v21-diff-for-0001 on the v20-0001 patch and
get the v21-0001 patch. There are a lot of changes, so I guess it will
help you during review. Please, let me know whether it is useful for you.
--
Best regards,
Daniil Davydov
Attachments:
[text/x-patch] v21-0005-Documentation-for-parallel-autovacuum.patch (4.4K, 2-v21-0005-Documentation-for-parallel-autovacuum.patch)
download | inline diff:
From ed32c12c05dac5fbfca55c7424f0c80d9d58fef5 Mon Sep 17 00:00:00 2001
From: Daniil Davidov <[email protected]>
Date: Sun, 23 Nov 2025 02:32:44 +0700
Subject: [PATCH v21 5/5] Documentation for parallel autovacuum
---
doc/src/sgml/config.sgml | 17 +++++++++++++++++
doc/src/sgml/maintenance.sgml | 12 ++++++++++++
doc/src/sgml/ref/create_table.sgml | 20 ++++++++++++++++++++
3 files changed, 49 insertions(+)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 37342986969..a6869b03753 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2886,6 +2886,7 @@ include_dir 'conf.d'
<para>
When changing this value, consider also adjusting
<xref linkend="guc-max-parallel-workers"/>,
+ <xref linkend="guc-autovacuum-max-parallel-workers"/>,
<xref linkend="guc-max-parallel-maintenance-workers"/>, and
<xref linkend="guc-max-parallel-workers-per-gather"/>.
</para>
@@ -9351,6 +9352,22 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-autovacuum-max-parallel-workers" xreflabel="autovacuum_max_parallel_workers">
+ <term><varname>autovacuum_max_parallel_workers</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>autovacuum_max_parallel_workers</varname></primary>
+ <secondary>configuration parameter</secondary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Sets the maximum number of parallel autovacuum workers that
+ can be used for parallel index vacuuming at one time. Is capped by
+ <xref linkend="guc-max-parallel-workers"/>. The default is 2.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 7c958b06273..c9f9163c551 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -926,6 +926,18 @@ HINT: Execute a database-wide VACUUM in that database.
autovacuum workers' activity.
</para>
+ <para>
+ If an autovacuum worker process comes across a table with the enabled
+ <xref linkend="reloption-autovacuum-parallel-workers"/> storage parameter,
+ it will launch parallel workers in order to vacuum indexes of this table
+ in a parallel mode. Parallel workers are taken from the pool of processes
+ established by <xref linkend="guc-max-worker-processes"/>, limited by
+ <xref linkend="guc-max-parallel-workers"/>.
+ The total number of parallel autovacuum workers that can be active at one
+ time is limited by the <xref linkend="guc-autovacuum-max-parallel-workers"/>
+ configuration parameter.
+ </para>
+
<para>
If several large tables all become eligible for vacuuming in a short
amount of time, all autovacuum workers might become occupied with
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 77c5a763d45..3592c9acff9 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -1717,6 +1717,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry id="reloption-autovacuum-parallel-workers" xreflabel="autovacuum_parallel_workers">
+ <term><literal>autovacuum_parallel_workers</literal> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>autovacuum_parallel_workers</varname> storage parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Sets the maximum number of parallel autovacuum workers that can process
+ indexes of this table.
+ The default value is -1, which means no parallel index vacuuming for
+ this table. If value is 0 then parallel degree will computed based on
+ number of indexes.
+ Note that the computed number of workers may not actually be available at
+ run time. If this occurs, the autovacuum will run with fewer workers
+ than expected.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="reloption-autovacuum-vacuum-threshold" xreflabel="autovacuum_vacuum_threshold">
<term><literal>autovacuum_vacuum_threshold</literal>, <literal>toast.autovacuum_vacuum_threshold</literal> (<type>integer</type>)
<indexterm>
--
2.43.0
[text/x-patch] v21-0001-Parallel-autovacuum.patch (19.4K, 3-v21-0001-Parallel-autovacuum.patch)
download | inline diff:
From 32064b095f6319033bb87e63c16c5f1e323bfe0f Mon Sep 17 00:00:00 2001
From: Daniil Davidov <[email protected]>
Date: Sun, 23 Nov 2025 01:03:24 +0700
Subject: [PATCH v21 1/5] Parallel autovacuum
---
src/backend/access/common/reloptions.c | 11 ++
src/backend/commands/vacuumparallel.c | 42 ++++-
src/backend/postmaster/autovacuum.c | 164 +++++++++++++++++-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc.c | 8 +-
src/backend/utils/misc/guc_parameters.dat | 8 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/bin/psql/tab-complete.in.c | 1 +
src/include/miscadmin.h | 1 +
src/include/postmaster/autovacuum.h | 5 +
src/include/utils/rel.h | 7 +
11 files changed, 239 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 237ab8d0ed9..9459a010cc3 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -235,6 +235,15 @@ static relopt_int intRelOpts[] =
},
SPGIST_DEFAULT_FILLFACTOR, SPGIST_MIN_FILLFACTOR, 100
},
+ {
+ {
+ "autovacuum_parallel_workers",
+ "Maximum number of parallel autovacuum workers that can be used for processing this table.",
+ RELOPT_KIND_HEAP,
+ ShareUpdateExclusiveLock
+ },
+ -1, -1, 1024
+ },
{
{
"autovacuum_vacuum_threshold",
@@ -1968,6 +1977,8 @@ default_reloptions(Datum reloptions, bool validate, relopt_kind kind)
{"fillfactor", RELOPT_TYPE_INT, offsetof(StdRdOptions, fillfactor)},
{"autovacuum_enabled", RELOPT_TYPE_BOOL,
offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, enabled)},
+ {"autovacuum_parallel_workers", RELOPT_TYPE_INT,
+ offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, autovacuum_parallel_workers)},
{"autovacuum_vacuum_threshold", RELOPT_TYPE_INT,
offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, vacuum_threshold)},
{"autovacuum_vacuum_max_threshold", RELOPT_TYPE_INT,
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index c3b3c9ea21a..d3e0c32b7ee 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1,7 +1,9 @@
/*-------------------------------------------------------------------------
*
* vacuumparallel.c
- * Support routines for parallel vacuum execution.
+ * Support routines for parallel vacuum and autovacuum execution. In the
+ * comments below, the word "vacuum" will refer to both vacuum and
+ * autovacuum.
*
* This file contains routines that are intended to support setting up, using,
* and tearing down a ParallelVacuumState.
@@ -34,6 +36,7 @@
#include "executor/instrument.h"
#include "optimizer/paths.h"
#include "pgstat.h"
+#include "postmaster/autovacuum.h"
#include "storage/bufmgr.h"
#include "tcop/tcopprot.h"
#include "utils/lsyscache.h"
@@ -373,8 +376,9 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes,
shared->queryid = pgstat_get_my_query_id();
shared->maintenance_work_mem_worker =
(nindexes_mwm > 0) ?
- maintenance_work_mem / Min(parallel_workers, nindexes_mwm) :
- maintenance_work_mem;
+ vac_work_mem / Min(parallel_workers, nindexes_mwm) :
+ vac_work_mem;
+
shared->dead_items_info.max_bytes = vac_work_mem * (size_t) 1024;
/* Prepare DSA space for dead items */
@@ -553,12 +557,17 @@ parallel_vacuum_compute_workers(Relation *indrels, int nindexes, int nrequested,
int nindexes_parallel_bulkdel = 0;
int nindexes_parallel_cleanup = 0;
int parallel_workers;
+ int max_workers;
+
+ max_workers = AmAutoVacuumWorkerProcess() ?
+ autovacuum_max_parallel_workers :
+ max_parallel_maintenance_workers;
/*
* We don't allow performing parallel operation in standalone backend or
* when parallelism is disabled.
*/
- if (!IsUnderPostmaster || max_parallel_maintenance_workers == 0)
+ if (!IsUnderPostmaster || max_workers == 0)
return 0;
/*
@@ -597,8 +606,8 @@ parallel_vacuum_compute_workers(Relation *indrels, int nindexes, int nrequested,
parallel_workers = (nrequested > 0) ?
Min(nrequested, nindexes_parallel) : nindexes_parallel;
- /* Cap by max_parallel_maintenance_workers */
- parallel_workers = Min(parallel_workers, max_parallel_maintenance_workers);
+ /* Cap by GUC variable */
+ parallel_workers = Min(parallel_workers, max_workers);
return parallel_workers;
}
@@ -646,6 +655,13 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
*/
nworkers = Min(nworkers, pvs->pcxt->nworkers);
+ /*
+ * Reserve workers in autovacuum global state. Note that we may be given
+ * fewer workers than we requested.
+ */
+ if (AmAutoVacuumWorkerProcess() && nworkers > 0)
+ AutoVacuumReserveParallelWorkers(&nworkers);
+
/*
* Set index vacuum status and mark whether parallel vacuum worker can
* process it.
@@ -690,6 +706,16 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
LaunchParallelWorkers(pvs->pcxt);
+ /*
+ * Tell autovacuum that we could not launch all the previously
+ * reserved workers.
+ */
+ if (AmAutoVacuumWorkerProcess() &&
+ pvs->pcxt->nworkers_launched < nworkers)
+ {
+ AutoVacuumReleaseParallelWorkers(nworkers - pvs->pcxt->nworkers_launched);
+ }
+
if (pvs->pcxt->nworkers_launched > 0)
{
/*
@@ -738,6 +764,10 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
for (int i = 0; i < pvs->pcxt->nworkers_launched; i++)
InstrAccumParallelQuery(&pvs->buffer_usage[i], &pvs->wal_usage[i]);
+
+ /* Release all the reserved parallel workers for autovacuum */
+ if (AmAutoVacuumWorkerProcess() && pvs->pcxt->nworkers_launched > 0)
+ AutoVacuumReleaseAllParallelWorkers();
}
/*
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 6fde740465f..f40abe90ed5 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -151,6 +151,13 @@ int Log_autoanalyze_min_duration = 600000;
static double av_storage_param_cost_delay = -1;
static int av_storage_param_cost_limit = -1;
+/*
+ * Tracks the number of parallel workers currently reserved by the
+ * autovacuum worker. This is non-zero only for the parallel autovacuum
+ * leader process.
+ */
+static int av_nworkers_reserved = 0;
+
/* Flags set by signal handlers */
static volatile sig_atomic_t got_SIGUSR2 = false;
@@ -285,6 +292,8 @@ typedef struct AutoVacuumWorkItem
* av_workItems work item array
* av_nworkersForBalance the number of autovacuum workers to use when
* calculating the per worker cost limit
+ * av_freeParallelWorkers the number of free parallel autovacuum workers
+ * av_maxParallelWorkers the maximum number of parallel autovacuum workers
*
* This struct is protected by AutovacuumLock, except for av_signal and parts
* of the worker list (see above).
@@ -299,6 +308,8 @@ typedef struct
WorkerInfo av_startingWorker;
AutoVacuumWorkItem av_workItems[NUM_WORKITEMS];
pg_atomic_uint32 av_nworkersForBalance;
+ uint32 av_freeParallelWorkers;
+ uint32 av_maxParallelWorkers;
} AutoVacuumShmemStruct;
static AutoVacuumShmemStruct *AutoVacuumShmem;
@@ -361,6 +372,7 @@ static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
static void avl_sigusr2_handler(SIGNAL_ARGS);
static bool av_worker_available(void);
static void check_av_worker_gucs(void);
+static void adjust_free_parallel_workers(int prev_max_parallel_workers);
@@ -759,6 +771,8 @@ ProcessAutoVacLauncherInterrupts(void)
if (ConfigReloadPending)
{
int autovacuum_max_workers_prev = autovacuum_max_workers;
+ int autovacuum_max_parallel_workers_prev =
+ autovacuum_max_parallel_workers;
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
@@ -775,6 +789,15 @@ ProcessAutoVacLauncherInterrupts(void)
if (autovacuum_max_workers_prev != autovacuum_max_workers)
check_av_worker_gucs();
+ /*
+ * If autovacuum_max_parallel_workers changed, we must take care of
+ * the correct value of available parallel autovacuum workers in
+ * shmem.
+ */
+ if (autovacuum_max_parallel_workers_prev !=
+ autovacuum_max_parallel_workers)
+ adjust_free_parallel_workers(autovacuum_max_parallel_workers_prev);
+
/* rebuild the list in case the naptime changed */
rebuild_database_list(InvalidOid);
}
@@ -1379,6 +1402,16 @@ avl_sigusr2_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
+/*
+ * Make sure that all reserved workers are released, even if parallel
+ * autovacuum leader is finishing due to FATAL error.
+ */
+static void
+autovacuum_worker_before_shmem_exit(int code, Datum arg)
+{
+ AutoVacuumReleaseAllParallelWorkers();
+}
+
/*
* Main entry point for autovacuum worker processes.
*/
@@ -2275,6 +2308,12 @@ do_autovacuum(void)
"Autovacuum Portal",
ALLOCSET_DEFAULT_SIZES);
+ /*
+ * Parallel autovacuum can reserve parallel workers. Make sure that all
+ * reserved workers are released even after FATAL error.
+ */
+ before_shmem_exit(autovacuum_worker_before_shmem_exit, 0);
+
/*
* Perform operations on collected tables.
*/
@@ -2456,6 +2495,12 @@ do_autovacuum(void)
}
PG_CATCH();
{
+ /*
+ * Parallel autovacuum can reserve parallel workers. Make sure
+ * that all reserved workers are released.
+ */
+ AutoVacuumReleaseAllParallelWorkers();
+
/*
* Abort the transaction, start a new one, and proceed with the
* next table in our list.
@@ -2856,8 +2901,12 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
*/
tab->at_params.index_cleanup = VACOPTVALUE_UNSPECIFIED;
tab->at_params.truncate = VACOPTVALUE_UNSPECIFIED;
- /* As of now, we don't support parallel vacuum for autovacuum */
- tab->at_params.nworkers = -1;
+
+ /* Decide whether we need to process indexes of table in parallel. */
+ tab->at_params.nworkers = avopts
+ ? avopts->autovacuum_parallel_workers
+ : -1;
+
tab->at_params.freeze_min_age = freeze_min_age;
tab->at_params.freeze_table_age = freeze_table_age;
tab->at_params.multixact_freeze_min_age = multixact_freeze_min_age;
@@ -3334,6 +3383,88 @@ AutoVacuumRequestWork(AutoVacuumWorkItemType type, Oid relationId,
return result;
}
+/*
+ * Reserves parallel workers for autovacuum.
+ *
+ * nworkers is an in/out parameter; the requested number of parallel workers
+ * to reserve by the caller, and set to the actual number of reserved workers.
+ *
+ * The caller must call AutoVacuumRelease[All]ParallelWorkers() to release the
+ * reserved workers.
+ *
+ * NOTE: We will try to provide as many workers as requested, even if caller
+ * will occupy all available workers.
+ */
+void
+AutoVacuumReserveParallelWorkers(int *nworkers)
+{
+ /* Only leader autovacuum worker can call this function. */
+ Assert(AmAutoVacuumWorkerProcess());
+
+ /* The worker must not have any reserved workers yet */
+ Assert(av_nworkers_reserved == 0);
+
+ LWLockAcquire(AutovacuumLock, LW_EXCLUSIVE);
+
+ /* Provide as many workers as we can. */
+ *nworkers = Min(AutoVacuumShmem->av_freeParallelWorkers, *nworkers);
+ AutoVacuumShmem->av_freeParallelWorkers -= *nworkers;
+
+ LWLockRelease(AutovacuumLock);
+
+ /* Remember how many workers we have reserved. */
+ av_nworkers_reserved = *nworkers;
+}
+
+/*
+ * Releases the reserved parallel workers for autovacuum.
+ *
+ * This function should be used to release the parallel workers that an
+ * autovacuum worker reserved by AutoVacuumReserveParallelWorkers(). nworkers
+ * is the number of workers to release, which must not be greater than the
+ * number of workers currently reserved, av_nworkers_reserved.
+ */
+void
+AutoVacuumReleaseParallelWorkers(int nworkers)
+{
+ /* Only leader worker can call this function. */
+ Assert(AmAutoVacuumWorkerProcess());
+
+ /* Cannot release more workers than reserved */
+ Assert(nworkers <= av_nworkers_reserved);
+
+ LWLockAcquire(AutovacuumLock, LW_EXCLUSIVE);
+
+ /*
+ * If the maximum number of parallel workers was reduced during execution,
+ * we must cap available workers number by its new value.
+ */
+ AutoVacuumShmem->av_freeParallelWorkers =
+ Min(AutoVacuumShmem->av_freeParallelWorkers + nworkers,
+ AutoVacuumShmem->av_maxParallelWorkers);
+
+ LWLockRelease(AutovacuumLock);
+
+ /* Don't have to remember these workers anymore. */
+ av_nworkers_reserved -= nworkers;
+}
+
+/*
+ * Same as above, but this function releases all the parallel workers that
+ * this autovacuum worker reserved.
+ */
+void
+AutoVacuumReleaseAllParallelWorkers(void)
+{
+ /* Only leader worker can call this function. */
+ Assert(AmAutoVacuumWorkerProcess());
+
+ if (av_nworkers_reserved > 0)
+ AutoVacuumReleaseParallelWorkers(av_nworkers_reserved);
+
+ Assert(av_nworkers_reserved == 0);
+}
+
/*
* autovac_init
* This is called at postmaster initialization.
@@ -3394,6 +3525,10 @@ AutoVacuumShmemInit(void)
Assert(!found);
AutoVacuumShmem->av_launcherpid = 0;
+ AutoVacuumShmem->av_maxParallelWorkers =
+ Min(autovacuum_max_parallel_workers, max_parallel_workers);
+ AutoVacuumShmem->av_freeParallelWorkers =
+ AutoVacuumShmem->av_maxParallelWorkers;
dclist_init(&AutoVacuumShmem->av_freeWorkers);
dlist_init(&AutoVacuumShmem->av_runningWorkers);
AutoVacuumShmem->av_startingWorker = NULL;
@@ -3475,3 +3610,28 @@ check_av_worker_gucs(void)
errdetail("The server will only start up to \"autovacuum_worker_slots\" (%d) autovacuum workers at a given time.",
autovacuum_worker_slots)));
}
+
+/*
+ * Adjusts the number of free parallel workers corresponds to the new
+ * autovacuum_max_parallel_workers value.
+ */
+static void
+adjust_free_parallel_workers(int prev_max_parallel_workers)
+{
+ int nfree_workers;
+
+ LWLockAcquire(AutovacuumLock, LW_EXCLUSIVE);
+
+ /*
+ * Cap or increase number of free parallel workers according to the
+ * parameter change.
+ */
+ nfree_workers =
+ autovacuum_max_parallel_workers - prev_max_parallel_workers +
+ AutoVacuumShmem->av_freeParallelWorkers;
+
+ AutoVacuumShmem->av_freeParallelWorkers = Max(nfree_workers, 0);
+ AutoVacuumShmem->av_maxParallelWorkers = autovacuum_max_parallel_workers;
+
+ LWLockRelease(AutovacuumLock);
+}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 36ad708b360..8265a82b639 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -143,6 +143,7 @@ int NBuffers = 16384;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
+int autovacuum_max_parallel_workers = 2;
int MaxBackends = 0;
/* GUC parameters for vacuum */
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index ae9d5f3fb70..c8a99a67767 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -3326,9 +3326,13 @@ set_config_with_handle(const char *name, config_handle *handle,
*
* Also allow normal setting if the GUC is marked GUC_ALLOW_IN_PARALLEL.
*
- * Other changes might need to affect other workers, so forbid them.
+ * Other changes might need to affect other workers, so forbid them. Note,
+ * that parallel autovacuum leader is an exception, because only
+ * cost-based delays need to be affected also to parallel vacuum workers,
+ * and we will handle it elsewhere if appropriate.
*/
- if (IsInParallelMode() && changeVal && action != GUC_ACTION_SAVE &&
+ if (IsInParallelMode() && !AmAutoVacuumWorkerProcess() && changeVal &&
+ action != GUC_ACTION_SAVE &&
(record->flags & GUC_ALLOW_IN_PARALLEL) == 0)
{
ereport(elevel,
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 762b8efe6b0..361bdb9a720 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -154,6 +154,14 @@
max => '2000000000',
},
+{ name => 'autovacuum_max_parallel_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
+ short_desc => 'Maximum number of parallel autovacuum workers, that can be taken from bgworkers pool.',
+ variable => 'autovacuum_max_parallel_workers',
+ boot_val => '2',
+ min => '0',
+ max => 'MAX_BACKENDS',
+},
+
{ name => 'autovacuum_max_workers', type => 'int', context => 'PGC_SIGHUP', group => 'VACUUM_AUTOVACUUM',
short_desc => 'Sets the maximum number of simultaneously running autovacuum worker processes.',
variable => 'autovacuum_max_workers',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 6e82c8e055d..2d4b9d27e8b 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -709,6 +709,7 @@
#autovacuum_worker_slots = 16 # autovacuum worker slots to allocate
# (change requires restart)
#autovacuum_max_workers = 3 # max number of autovacuum subprocesses
+#autovacuum_max_parallel_workers = 2 # limited by max_parallel_workers
#autovacuum_naptime = 1min # time between autovacuum runs
#autovacuum_vacuum_threshold = 50 # min number of row updates before
# vacuum
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 8b91bc00062..ed59a21289c 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -1423,6 +1423,7 @@ static const char *const table_storage_parameters[] = {
"autovacuum_multixact_freeze_max_age",
"autovacuum_multixact_freeze_min_age",
"autovacuum_multixact_freeze_table_age",
+ "autovacuum_parallel_workers",
"autovacuum_vacuum_cost_delay",
"autovacuum_vacuum_cost_limit",
"autovacuum_vacuum_insert_scale_factor",
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index db559b39c4d..ad6e19f426c 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -178,6 +178,7 @@ extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
+extern PGDLLIMPORT int autovacuum_max_parallel_workers;
extern PGDLLIMPORT int commit_timestamp_buffers;
extern PGDLLIMPORT int multixact_member_buffers;
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 5aa0f3a8ac1..f3783afb51b 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -62,6 +62,11 @@ pg_noreturn extern void AutoVacWorkerMain(const void *startup_data, size_t start
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
Oid relationId, BlockNumber blkno);
+/* parallel autovacuum stuff */
+extern void AutoVacuumReserveParallelWorkers(int *nworkers);
+extern void AutoVacuumReleaseParallelWorkers(int nworkers);
+extern void AutoVacuumReleaseAllParallelWorkers(void);
+
/* shared memory stuff */
extern Size AutoVacuumShmemSize(void);
extern void AutoVacuumShmemInit(void);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 236830f6b93..7c5e35a486c 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -311,6 +311,13 @@ typedef struct ForeignKeyCacheInfo
typedef struct AutoVacOpts
{
bool enabled;
+
+ /*
+ * Max number of parallel autovacuum workers. If value is 0 then parallel
+ * degree will computed based on number of indexes.
+ */
+ int autovacuum_parallel_workers;
+
int vacuum_threshold;
int vacuum_max_threshold;
int vacuum_ins_threshold;
--
2.43.0
[text/x-patch] v21-0002-Logging-for-parallel-autovacuum.patch (10.2K, 4-v21-0002-Logging-for-parallel-autovacuum.patch)
download | inline diff:
From ba2a21114126d6c2b9ea8629a7299332e573136a Mon Sep 17 00:00:00 2001
From: Daniil Davidov <[email protected]>
Date: Sun, 23 Nov 2025 01:07:47 +0700
Subject: [PATCH v21 2/5] Logging for parallel autovacuum
---
src/backend/access/heap/vacuumlazy.c | 61 ++++++++++++++++++++++++++-
src/backend/commands/vacuumparallel.c | 29 ++++++++++---
src/include/commands/vacuum.h | 28 +++++++++++-
src/tools/pgindent/typedefs.list | 3 ++
4 files changed, 111 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 4be267ff657..d19e15cbcce 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -340,6 +340,12 @@ typedef struct LVRelState
int num_index_scans;
int num_dead_items_resets;
Size total_dead_items_bytes;
+
+ /*
+ * Total number of planned and actually launched parallel workers for
+ * index scans.
+ */
+ PVWorkersUsage workers_usage;
/* Counters that follow are only for scanned_pages */
int64 tuples_deleted; /* # deleted from table */
int64 tuples_frozen; /* # newly frozen */
@@ -778,6 +784,11 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->vm_new_visible_frozen_pages = 0;
vacrel->vm_new_frozen_pages = 0;
+ vacrel->workers_usage.vacuum.nlaunched = 0;
+ vacrel->workers_usage.vacuum.nplanned = 0;
+ vacrel->workers_usage.cleanup.nlaunched = 0;
+ vacrel->workers_usage.cleanup.nplanned = 0;
+
/*
* Get cutoffs that determine which deleted tuples are considered DEAD,
* not just RECENTLY_DEAD, and which XIDs/MXIDs to freeze. Then determine
@@ -1120,6 +1131,50 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
orig_rel_pages == 0 ? 100.0 :
100.0 * vacrel->lpdead_item_pages / orig_rel_pages,
vacrel->lpdead_items);
+ if (vacrel->workers_usage.vacuum.nplanned > 0)
+ {
+ /* Stats for vacuum phase of index vacuuming. */
+
+ if (AmAutoVacuumWorkerProcess())
+ {
+ /* Worker usage stats for parallel autovacuum. */
+ appendStringInfo(&buf,
+ _("parallel index vacuum: %d workers were planned, %d workers were reserved and %d workers were launched in total\n"),
+ vacrel->workers_usage.vacuum.nplanned,
+ vacrel->workers_usage.vacuum.nreserved,
+ vacrel->workers_usage.vacuum.nlaunched);
+ }
+ else
+ {
+ /* Worker usage stats for manual VACUUM (PARALLEL). */
+ appendStringInfo(&buf,
+ _("parallel index vacuum: %d workers were planned and %d workers were launched in total\n"),
+ vacrel->workers_usage.vacuum.nplanned,
+ vacrel->workers_usage.vacuum.nlaunched);
+ }
+ }
+ if (vacrel->workers_usage.cleanup.nplanned > 0)
+ {
+ /* Stats for cleanup phase of index vacuuming. */
+
+ if (AmAutoVacuumWorkerProcess())
+ {
+ /* Worker usage stats for parallel autovacuum. */
+ appendStringInfo(&buf,
+ _("parallel index cleanup: %d workers were planned, %d workers were reserved and %d workers were launched in total\n"),
+ vacrel->workers_usage.cleanup.nplanned,
+ vacrel->workers_usage.cleanup.nreserved,
+ vacrel->workers_usage.cleanup.nlaunched);
+ }
+ else
+ {
+ /* Worker usage stats for manual VACUUM (PARALLEL). */
+ appendStringInfo(&buf,
+ _("parallel index cleanup: %d workers were planned and %d workers were launched in total\n"),
+ vacrel->workers_usage.cleanup.nplanned,
+ vacrel->workers_usage.cleanup.nlaunched);
+ }
+ }
for (int i = 0; i < vacrel->nindexes; i++)
{
IndexBulkDeleteResult *istat = vacrel->indstats[i];
@@ -2664,7 +2719,8 @@ lazy_vacuum_all_indexes(LVRelState *vacrel)
{
/* Outsource everything to parallel variant */
parallel_vacuum_bulkdel_all_indexes(vacrel->pvs, old_live_tuples,
- vacrel->num_index_scans);
+ vacrel->num_index_scans,
+ &vacrel->workers_usage);
/*
* Do a postcheck to consider applying wraparound failsafe now. Note
@@ -3097,7 +3153,8 @@ lazy_cleanup_all_indexes(LVRelState *vacrel)
/* Outsource everything to parallel variant */
parallel_vacuum_cleanup_all_indexes(vacrel->pvs, reltuples,
vacrel->num_index_scans,
- estimated_count);
+ estimated_count,
+ &vacrel->workers_usage);
}
/* Reset the progress counters */
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index d3e0c32b7ee..86d9f2b74c9 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -227,7 +227,7 @@ struct ParallelVacuumState
static int parallel_vacuum_compute_workers(Relation *indrels, int nindexes, int nrequested,
bool *will_parallel_vacuum);
static void parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scans,
- bool vacuum);
+ bool vacuum, PVWorkersStats *wstats);
static void parallel_vacuum_process_safe_indexes(ParallelVacuumState *pvs);
static void parallel_vacuum_process_unsafe_indexes(ParallelVacuumState *pvs);
static void parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
@@ -502,7 +502,7 @@ parallel_vacuum_reset_dead_items(ParallelVacuumState *pvs)
*/
void
parallel_vacuum_bulkdel_all_indexes(ParallelVacuumState *pvs, long num_table_tuples,
- int num_index_scans)
+ int num_index_scans, PVWorkersUsage *wusage)
{
Assert(!IsParallelWorker());
@@ -513,7 +513,8 @@ parallel_vacuum_bulkdel_all_indexes(ParallelVacuumState *pvs, long num_table_tup
pvs->shared->reltuples = num_table_tuples;
pvs->shared->estimated_count = true;
- parallel_vacuum_process_all_indexes(pvs, num_index_scans, true);
+ parallel_vacuum_process_all_indexes(pvs, num_index_scans, true,
+ &wusage->vacuum);
}
/*
@@ -521,7 +522,8 @@ parallel_vacuum_bulkdel_all_indexes(ParallelVacuumState *pvs, long num_table_tup
*/
void
parallel_vacuum_cleanup_all_indexes(ParallelVacuumState *pvs, long num_table_tuples,
- int num_index_scans, bool estimated_count)
+ int num_index_scans, bool estimated_count,
+ PVWorkersUsage *wusage)
{
Assert(!IsParallelWorker());
@@ -533,7 +535,8 @@ parallel_vacuum_cleanup_all_indexes(ParallelVacuumState *pvs, long num_table_tup
pvs->shared->reltuples = num_table_tuples;
pvs->shared->estimated_count = estimated_count;
- parallel_vacuum_process_all_indexes(pvs, num_index_scans, false);
+ parallel_vacuum_process_all_indexes(pvs, num_index_scans, false,
+ &wusage->cleanup);
}
/*
@@ -618,7 +621,7 @@ parallel_vacuum_compute_workers(Relation *indrels, int nindexes, int nrequested,
*/
static void
parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scans,
- bool vacuum)
+ bool vacuum, PVWorkersStats *wstats)
{
int nworkers;
PVIndVacStatus new_status;
@@ -655,13 +658,23 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
*/
nworkers = Min(nworkers, pvs->pcxt->nworkers);
+ /* Remember this value, if we asked to */
+ if (wstats != NULL && nworkers > 0)
+ wstats->nplanned += nworkers;
+
/*
* Reserve workers in autovacuum global state. Note that we may be given
* fewer workers than we requested.
*/
if (AmAutoVacuumWorkerProcess() && nworkers > 0)
+ {
AutoVacuumReserveParallelWorkers(&nworkers);
+ /* Remember this value, if we asked to */
+ if (wstats != NULL)
+ wstats->nreserved += nworkers;
+ }
+
/*
* Set index vacuum status and mark whether parallel vacuum worker can
* process it.
@@ -728,6 +741,10 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
/* Enable shared cost balance for leader backend */
VacuumSharedCostBalance = &(pvs->shared->cost_balance);
VacuumActiveNWorkers = &(pvs->shared->active_nworkers);
+
+ /* Remember this value, if we asked to */
+ if (wstats != NULL)
+ wstats->nlaunched += pvs->pcxt->nworkers_launched;
}
if (vacuum)
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index e885a4b9c77..d3dc4e8cc67 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -300,6 +300,28 @@ typedef struct VacDeadItemsInfo
int64 num_items; /* current # of entries */
} VacDeadItemsInfo;
+/*
+ * Helper for the PVWorkersUsage structure (see below), to avoid repetition.
+ */
+typedef struct PVWorkersStats
+{
+ int nplanned; /* # of parallel workers we are planned to
+ * launch */
+ int nreserved; /* for autovacuum only - # of parallel workers
+ * we have managed to reserve */
+ int nlaunched; /* # of launched parallel workers */
+} PVWorkersStats;
+
+/*
+ * PVWorkersUsage stores information about total number of launched, reserved
+ * and planned workers during parallel vacuum (both for vacuum and cleanup).
+ */
+typedef struct PVWorkersUsage
+{
+ PVWorkersStats vacuum;
+ PVWorkersStats cleanup;
+} PVWorkersUsage;
+
/* GUC parameters */
extern PGDLLIMPORT int default_statistics_target; /* PGDLLIMPORT for PostGIS */
extern PGDLLIMPORT int vacuum_freeze_min_age;
@@ -394,11 +416,13 @@ extern TidStore *parallel_vacuum_get_dead_items(ParallelVacuumState *pvs,
extern void parallel_vacuum_reset_dead_items(ParallelVacuumState *pvs);
extern void parallel_vacuum_bulkdel_all_indexes(ParallelVacuumState *pvs,
long num_table_tuples,
- int num_index_scans);
+ int num_index_scans,
+ PVWorkersUsage *wusage);
extern void parallel_vacuum_cleanup_all_indexes(ParallelVacuumState *pvs,
long num_table_tuples,
int num_index_scans,
- bool estimated_count);
+ bool estimated_count,
+ PVWorkersUsage *wusage);
extern void parallel_vacuum_main(dsm_segment *seg, shm_toc *toc);
/* in commands/analyze.c */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index df42b78bc9d..d84308c87ad 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2067,6 +2067,8 @@ PVIndStats
PVIndVacStatus
PVOID
PVShared
+PVWorkersUsage
+PVWorkersStats
PX_Alias
PX_Cipher
PX_Combo
@@ -2405,6 +2407,7 @@ PullFilterOps
PushFilter
PushFilterOps
PushFunction
+PVWorkersUsage
PyCFunction
PyMethodDef
PyModuleDef
--
2.43.0
[text/x-patch] v21-0004-Tests-for-parallel-autovacuum.patch (23.3K, 5-v21-0004-Tests-for-parallel-autovacuum.patch)
download | inline diff:
From 6a9900a79e72e7e5366265ae074939742006ea08 Mon Sep 17 00:00:00 2001
From: Daniil Davidov <[email protected]>
Date: Sun, 23 Nov 2025 01:08:14 +0700
Subject: [PATCH v21 4/5] Tests for parallel autovacuum
---
src/backend/access/heap/vacuumlazy.c | 7 +
src/backend/commands/vacuumparallel.c | 55 +++
src/backend/postmaster/autovacuum.c | 28 ++
src/include/postmaster/autovacuum.h | 1 +
src/test/modules/Makefile | 1 +
src/test/modules/meson.build | 1 +
src/test/modules/test_autovacuum/.gitignore | 2 +
src/test/modules/test_autovacuum/Makefile | 28 ++
src/test/modules/test_autovacuum/meson.build | 36 ++
.../modules/test_autovacuum/t/001_basic.pl | 332 ++++++++++++++++++
.../test_autovacuum/test_autovacuum--1.0.sql | 12 +
.../modules/test_autovacuum/test_autovacuum.c | 41 +++
.../test_autovacuum/test_autovacuum.control | 3 +
13 files changed, 547 insertions(+)
create mode 100644 src/test/modules/test_autovacuum/.gitignore
create mode 100644 src/test/modules/test_autovacuum/Makefile
create mode 100644 src/test/modules/test_autovacuum/meson.build
create mode 100644 src/test/modules/test_autovacuum/t/001_basic.pl
create mode 100644 src/test/modules/test_autovacuum/test_autovacuum--1.0.sql
create mode 100644 src/test/modules/test_autovacuum/test_autovacuum.c
create mode 100644 src/test/modules/test_autovacuum/test_autovacuum.control
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d19e15cbcce..2e85f7f17f7 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -151,6 +151,7 @@
#include "storage/freespace.h"
#include "storage/lmgr.h"
#include "storage/read_stream.h"
+#include "utils/injection_point.h"
#include "utils/lsyscache.h"
#include "utils/pg_rusage.h"
#include "utils/timestamp.h"
@@ -869,6 +870,12 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
lazy_check_wraparound_failsafe(vacrel);
dead_items_alloc(vacrel, params.nworkers);
+ /*
+ * Trigger injection point, if parallel autovacuum is about to be started.
+ */
+ if (AmAutoVacuumWorkerProcess() && ParallelVacuumIsActive(vacrel))
+ INJECTION_POINT("autovacuum-start-parallel-vacuum", NULL);
+
/*
* Call lazy_scan_heap to perform all required heap pruning, index
* vacuuming, and heap vacuuming (plus related processing)
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index ccb3812165c..f9f29f940c9 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -39,6 +39,7 @@
#include "postmaster/autovacuum.h"
#include "storage/bufmgr.h"
#include "tcop/tcopprot.h"
+#include "utils/injection_point.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -303,6 +304,10 @@ static bool parallel_vacuum_index_is_parallel_safe(Relation indrel, int num_inde
bool vacuum);
static void parallel_vacuum_error_callback(void *arg);
+#ifdef USE_INJECTION_POINTS
+static void parallel_vacuum_report_cost_based_params(void);
+#endif
+
/*
* Try to enter parallel mode and create a parallel context. Then initialize
* shared memory state.
@@ -922,6 +927,17 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
pvs->pcxt->nworkers_launched, nworkers)));
}
+ /*
+ * To be able to exercise whether all reserved parallel workers are being
+ * released anyway, allow injection points to trigger a failure at this
+ * point.
+ *
+ * This injection point is also used to wait until parallel workers
+ * finishes their part of index processing.
+ */
+ if (nworkers > 0)
+ INJECTION_POINT("autovacuum-leader-before-indexes-processing", NULL);
+
/* Vacuum the indexes that can be processed by only leader process */
parallel_vacuum_process_unsafe_indexes(pvs);
@@ -1299,6 +1315,16 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Process indexes to perform vacuum/cleanup */
parallel_vacuum_process_safe_indexes(&pvs);
+#ifdef USE_INJECTION_POINTS
+ /*
+ * If we are parallel autovacuum worker, we can consume delay parameters
+ * during index processing (via vacuum_delay_point call). This logging
+ * allows tests to ensure this.
+ */
+ if (shared->am_parallel_autovacuum)
+ parallel_vacuum_report_cost_based_params();
+#endif
+
/* Report buffer/WAL usage during parallel execution */
buffer_usage = shm_toc_lookup(toc, PARALLEL_VACUUM_KEY_BUFFER_USAGE, false);
wal_usage = shm_toc_lookup(toc, PARALLEL_VACUUM_KEY_WAL_USAGE, false);
@@ -1351,3 +1377,32 @@ parallel_vacuum_error_callback(void *arg)
return;
}
}
+
+#ifdef USE_INJECTION_POINTS
+/*
+ * Log values of the related to cost-based delay parameters. It is used for
+ * testing purpose.
+ */
+static void
+parallel_vacuum_report_cost_based_params(void)
+{
+ StringInfoData buf;
+
+ /* Simulate config reload during normal processing */
+ pg_atomic_add_fetch_u32(VacuumActiveNWorkers, 1);
+ vacuum_delay_point(false);
+ pg_atomic_sub_fetch_u32(VacuumActiveNWorkers, 1);
+
+ initStringInfo(&buf);
+
+ appendStringInfo(&buf, "Vacuum cost-based delay parameters of parallel worker:\n");
+ appendStringInfo(&buf, "vacuum_cost_limit = %d\n",vacuum_cost_limit);
+ appendStringInfo(&buf, "vacuum_cost_delay = %g\n", vacuum_cost_delay);
+ appendStringInfo(&buf, "vacuum_cost_page_miss = %d\n", VacuumCostPageMiss);
+ appendStringInfo(&buf, "vacuum_cost_page_dirty = %d\n", VacuumCostPageDirty);
+ appendStringInfo(&buf, "vacuum_cost_page_hit = %d\n", VacuumCostPageHit);
+
+ ereport(DEBUG2, errmsg("%s", buf.data));
+ pfree(buf.data);
+}
+#endif
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 0d78d02bd09..7b24a5d6e67 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -2495,12 +2495,20 @@ do_autovacuum(void)
}
PG_CATCH();
{
+ int nreserved_workers = av_nworkers_reserved;
+
/*
* Parallel autovacuum can reserve parallel workers. Make sure
* that all reserved workers are released.
*/
AutoVacuumReleaseAllParallelWorkers();
+ if (nreserved_workers > 0)
+ ereport(DEBUG2,
+ (errmsg("%d parallel autovacuum workers has been released after occured error",
+ nreserved_workers),
+ errhidecontext(true)));
+
/*
* Abort the transaction, start a new one, and proceed with the
* next table in our list.
@@ -3465,6 +3473,21 @@ AutoVacuumReleaseAllParallelWorkers(void)
Assert(av_nworkers_reserved == 0);
}
+/*
+ * Get number of free autovacuum parallel workers.
+ */
+uint32
+AutoVacuumGetFreeParallelWorkers(void)
+{
+ uint32 nfree_workers;
+
+ LWLockAcquire(AutovacuumLock, LW_SHARED);
+ nfree_workers = AutoVacuumShmem->av_freeParallelWorkers;
+ LWLockRelease(AutovacuumLock);
+
+ return nfree_workers;
+}
+
/*
* autovac_init
* This is called at postmaster initialization.
@@ -3633,5 +3656,10 @@ adjust_free_parallel_workers(int prev_max_parallel_workers)
AutoVacuumShmem->av_freeParallelWorkers = Max(nfree_workers, 0);
AutoVacuumShmem->av_maxParallelWorkers = autovacuum_max_parallel_workers;
+ ereport(DEBUG2,
+ (errmsg("number of free parallel autovacuum workers is set to %u due to config reload",
+ AutoVacuumShmem->av_freeParallelWorkers),
+ errhidecontext(true)));
+
LWLockRelease(AutovacuumLock);
}
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index f3783afb51b..52be260e15f 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -66,6 +66,7 @@ extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
extern void AutoVacuumReserveParallelWorkers(int *nworkers);
extern void AutoVacuumReleaseParallelWorkers(int nworkers);
extern void AutoVacuumReleaseAllParallelWorkers(void);
+extern uint32 AutoVacuumGetFreeParallelWorkers(void);
/* shared memory stuff */
extern Size AutoVacuumShmemSize(void);
diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index 44c7163c1cd..937dbb64fd2 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -16,6 +16,7 @@ SUBDIRS = \
plsample \
spgist_name_ops \
test_aio \
+ test_autovacuum \
test_binaryheap \
test_bitmapset \
test_bloomfilter \
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 2634a519935..5ac8d87702d 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -16,6 +16,7 @@ subdir('plsample')
subdir('spgist_name_ops')
subdir('ssl_passphrase_callback')
subdir('test_aio')
+subdir('test_autovacuum')
subdir('test_binaryheap')
subdir('test_bitmapset')
subdir('test_bloomfilter')
diff --git a/src/test/modules/test_autovacuum/.gitignore b/src/test/modules/test_autovacuum/.gitignore
new file mode 100644
index 00000000000..716e17f5a2a
--- /dev/null
+++ b/src/test/modules/test_autovacuum/.gitignore
@@ -0,0 +1,2 @@
+# Generated subdirectories
+/tmp_check/
diff --git a/src/test/modules/test_autovacuum/Makefile b/src/test/modules/test_autovacuum/Makefile
new file mode 100644
index 00000000000..32254c53a5d
--- /dev/null
+++ b/src/test/modules/test_autovacuum/Makefile
@@ -0,0 +1,28 @@
+# src/test/modules/test_autovacuum/Makefile
+
+PGFILEDESC = "test_autovacuum - test code for parallel autovacuum"
+
+MODULE_big = test_autovacuum
+OBJS = \
+ $(WIN32RES) \
+ test_autovacuum.o
+
+EXTENSION = test_autovacuum
+DATA = test_autovacuum--1.0.sql
+
+TAP_TESTS = 1
+
+EXTRA_INSTALL = src/test/modules/injection_points
+
+export enable_injection_points
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_autovacuum
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/test_autovacuum/meson.build b/src/test/modules/test_autovacuum/meson.build
new file mode 100644
index 00000000000..3441e5e49cf
--- /dev/null
+++ b/src/test/modules/test_autovacuum/meson.build
@@ -0,0 +1,36 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+test_autovacuum_sources = files(
+ 'test_autovacuum.c',
+)
+
+if host_system == 'windows'
+ test_autovacuum_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'test_autovacuum',
+ '--FILEDESC', 'test_autovacuum - test code for parallel autovacuum',])
+endif
+
+test_autovacuum = shared_module('test_autovacuum',
+ test_autovacuum_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += test_autovacuum
+
+test_install_data += files(
+ 'test_autovacuum.control',
+ 'test_autovacuum--1.0.sql',
+)
+
+tests += {
+ 'name': 'test_autovacuum',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'tap': {
+ 'env': {
+ 'enable_injection_points': get_option('injection_points') ? 'yes' : 'no',
+ },
+ 'tests': [
+ 't/001_basic.pl',
+ ],
+ },
+}
diff --git a/src/test/modules/test_autovacuum/t/001_basic.pl b/src/test/modules/test_autovacuum/t/001_basic.pl
new file mode 100644
index 00000000000..b3d22361dcf
--- /dev/null
+++ b/src/test/modules/test_autovacuum/t/001_basic.pl
@@ -0,0 +1,332 @@
+# Test parallel autovacuum behavior
+
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+ plan skip_all => 'Injection points not supported by this build';
+}
+
+# Before each test we should disable autovacuum for 'test_autovac' table and
+# generate some dead tuples in it.
+
+sub prepare_for_next_test
+{
+ my ($node, $test_number) = @_;
+
+ $node->safe_psql('postgres', qq{
+ ALTER TABLE test_autovac SET (autovacuum_enabled = false);
+ });
+
+ $node->safe_psql('postgres', qq{
+ UPDATE test_autovac SET col_1 = $test_number;
+ });
+}
+
+
+my $psql_out;
+
+my $node = PostgreSQL::Test::Cluster->new('node1');
+$node->init;
+
+# Configure postgres, so it can launch parallel autovacuum workers, log all
+# information we are interested in and autovacuum works frequently
+$node->append_conf('postgresql.conf', qq{
+ max_worker_processes = 20
+ max_parallel_workers = 20
+ max_parallel_maintenance_workers = 20
+ autovacuum_max_parallel_workers = 20
+ log_min_messages = debug2
+ log_autovacuum_min_duration = 0
+ autovacuum_naptime = '1s'
+ min_parallel_index_scan_size = 0
+ shared_preload_libraries=test_autovacuum
+});
+$node->start;
+
+# Check if the extension injection_points is available, as it may be
+# possible that this script is run with installcheck, where the module
+# would not be installed by default.
+if (!$node->check_extension('injection_points'))
+{
+ plan skip_all => 'Extension injection_points not installed';
+}
+
+# Create all functions needed for testing
+$node->safe_psql('postgres', qq{
+ CREATE EXTENSION test_autovacuum;
+ CREATE EXTENSION injection_points;
+});
+
+my $indexes_num = 4;
+my $initial_rows_num = 10_000;
+my $autovacuum_parallel_workers = 2;
+
+# Create table and fill it with some data
+$node->safe_psql('postgres', qq{
+ CREATE TABLE test_autovac (
+ id SERIAL PRIMARY KEY,
+ col_1 INTEGER, col_2 INTEGER, col_3 INTEGER, col_4 INTEGER
+ ) WITH (autovacuum_parallel_workers = $autovacuum_parallel_workers);
+
+ INSERT INTO test_autovac
+ SELECT
+ g AS col1,
+ g + 1 AS col2,
+ g + 2 AS col3,
+ g + 3 AS col4
+ FROM generate_series(1, $initial_rows_num) AS g;
+});
+
+# Create specified number of b-tree indexes on the table
+$node->safe_psql('postgres', qq{
+ DO \$\$
+ DECLARE
+ i INTEGER;
+ BEGIN
+ FOR i IN 1..$indexes_num LOOP
+ EXECUTE format('CREATE INDEX idx_col_\%s ON test_autovac (col_\%s);', i, i);
+ END LOOP;
+ END \$\$;
+});
+
+# Test 1 :
+# Our table has enough indexes and appropriate reloptions, so autovacuum must
+# be able to process it in parallel mode. Just check if it can.
+# Also check whether all requested workers:
+# 1) launched
+# 2) correctly released
+
+prepare_for_next_test($node, 1);
+
+$node->safe_psql('postgres', qq{
+ ALTER TABLE test_autovac SET (autovacuum_enabled = true);
+});
+
+# Wait until the parallel autovacuum on table is completed. At the same time,
+# we check that the required number of parallel workers has been started.
+$log_start = $node->wait_for_log(
+ qr/parallel index vacuum: 2 workers were planned, / .
+ qr/2 workers were reserved and 2 workers were launched in total/,
+ $log_start
+);
+
+$psql_out = $node->safe_psql('postgres', qq{
+ SELECT get_parallel_autovacuum_free_workers();
+});
+is($psql_out, 20, 'All parallel workers has been released by the leader');
+
+# Test 2:
+# Check whether parallel autovacuum leader can propagate cost-based parameters
+# to parallel workers.
+
+prepare_for_next_test($node, 2);
+
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_attach('autovacuum-start-parallel-vacuum', 'wait');
+ SELECT injection_points_attach('autovacuum-leader-before-indexes-processing', 'wait');
+
+ ALTER TABLE test_autovac SET (autovacuum_parallel_workers = 1, autovacuum_enabled = true);
+});
+
+# Wait until parallel autovacuum is inited
+$node->wait_for_event(
+ 'autovacuum worker',
+ 'autovacuum-start-parallel-vacuum'
+);
+
+# Reload config - leader worker must update its own parameters during indexes
+# processing
+$node->safe_psql('postgres', qq{
+ ALTER SYSTEM SET vacuum_cost_limit = 500;
+ ALTER SYSTEM SET vacuum_cost_page_miss = 10;
+ ALTER SYSTEM SET vacuum_cost_page_dirty = 10;
+ ALTER SYSTEM SET vacuum_cost_page_hit = 10;
+ SELECT pg_reload_conf();
+});
+
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_wakeup('autovacuum-start-parallel-vacuum');
+});
+
+# Now wait until parallel autovacuum leader completes processing table (i.e.
+# guaranteed to call vacuum_delay_point) and launches parallel worker.
+$node->wait_for_event(
+ 'autovacuum worker',
+ 'autovacuum-leader-before-indexes-processing'
+);
+
+# Check whether parallel worker successfully updated all parameters during
+# index processing
+$log_start = $node->wait_for_log(
+ qr/Vacuum cost-based delay parameters of parallel worker:\n/ .
+ qr/\tvacuum_cost_limit = 500\n/ .
+ qr/\tvacuum_cost_delay = 2\n/ .
+ qr/\tvacuum_cost_page_miss = 10\n/ .
+ qr/\tvacuum_cost_page_dirty = 10\n/ .
+ qr/\tvacuum_cost_page_hit = 10\n/,
+ $log_start
+);
+
+# Cleanup
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_wakeup('autovacuum-leader-before-indexes-processing');
+
+ SELECT injection_points_detach('autovacuum-start-parallel-vacuum');
+ SELECT injection_points_detach('autovacuum-leader-before-indexes-processing');
+
+ ALTER TABLE test_autovac SET (autovacuum_parallel_workers = $autovacuum_parallel_workers);
+});
+
+# Test 3:
+# Test adjustment of free parallel workers number when changing
+# autovacuum_max_parallel_workers parameter
+
+prepare_for_next_test($node, 4);
+
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_attach('autovacuum-leader-before-indexes-processing', 'wait');
+ ALTER TABLE test_autovac SET (autovacuum_enabled = true);
+});
+
+$node->wait_for_event(
+ 'autovacuum worker',
+ 'autovacuum-leader-before-indexes-processing'
+);
+
+$node->safe_psql('postgres', qq{
+ ALTER SYSTEM SET autovacuum_max_parallel_workers = 1;
+ SELECT pg_reload_conf();
+});
+
+# Since 2 parallel workers already launched and will be released in the future,
+# we are expecting that :
+# 1) number of free workers will be '0' after config reload
+# 2) number of free workers will be '1' after releasing workers
+
+# Check statement (1)
+$log_start = $node->wait_for_log(
+ qr/number of free parallel autovacuum workers is set to 0 due to config reload/,
+ $log_start
+);
+
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_wakeup('autovacuum-leader-before-indexes-processing');
+});
+
+# Wait until the end of parallel processing
+$log_start = $node->wait_for_log(
+ qr/parallel index vacuum: 2 workers were planned, / .
+ qr/2 workers were reserved and 2 workers were launched in total/,
+ $log_start
+);
+
+# Check statement (2)
+$psql_out = $node->safe_psql('postgres', qq{
+ SELECT get_parallel_autovacuum_free_workers();
+});
+is($psql_out, 1, 'Number of free parallel workers is consistent');
+
+# Cleanup
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_detach('autovacuum-leader-before-indexes-processing');
+ ALTER SYSTEM SET autovacuum_max_parallel_workers = 10;
+ SELECT pg_reload_conf();
+});
+
+# Test 4:
+# We want parallel autovacuum workers to be released even if leader gets an
+# error. At first, simulate situation, when leader exits due to an ERROR.
+
+prepare_for_next_test($node, 4);
+
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_attach('autovacuum-leader-before-indexes-processing', 'error');
+ ALTER TABLE test_autovac SET (autovacuum_enabled = true);
+});
+
+$log_start = $node->wait_for_log(
+ qr/error triggered for injection point / .
+ qr/autovacuum-leader-before-indexes-processing/,
+ $log_start
+);
+
+$log_start = $node->wait_for_log(
+ qr/2 parallel autovacuum workers has been released after occured error/,
+ $log_start
+);
+
+# Cleanup
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_detach('autovacuum-leader-before-indexes-processing');
+});
+
+# Test 5:
+# Same as above test, but simulate situation, when leader exits due to FATAL.
+
+prepare_for_next_test($node, 5);
+
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_attach('autovacuum-start-parallel-vacuum', 'wait');
+ SELECT injection_points_attach('autovacuum-leader-before-indexes-processing', 'wait');
+ ALTER TABLE test_autovac SET (autovacuum_enabled = true);
+});
+
+# Wait until parallel autovacuum is inited and wake up the leader
+$node->wait_for_event(
+ 'autovacuum worker',
+ 'autovacuum-start-parallel-vacuum'
+);
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_wakeup('autovacuum-start-parallel-vacuum');
+});
+
+$node->wait_for_event(
+ 'autovacuum worker',
+ 'autovacuum-leader-before-indexes-processing'
+);
+
+my $av_pid = $node->safe_psql('postgres', qq{
+ SELECT pid FROM pg_stat_activity
+ WHERE backend_type = 'autovacuum worker'
+ AND wait_event = 'autovacuum-leader-before-indexes-processing'
+ LIMIT 1;
+});
+
+# Create role with pg_signal_autovacuum_worker for terminating autovacuum worker.
+$node->safe_psql('postgres', qq{
+ CREATE ROLE regress_worker_role;
+ GRANT pg_signal_autovacuum_worker TO regress_worker_role;
+ SET ROLE regress_worker_role;
+});
+
+$node->safe_psql('postgres', qq{
+ SELECT pg_terminate_backend('$av_pid');
+});
+
+$log_start = $node->wait_for_log(
+ qr/terminating autovacuum process due to administrator command/,
+ $log_start
+);
+
+# Now it is safe to check the number of free parallel workers, because even if
+# autovacuum is trying to vacuum table in parallel mode again, the leader
+# worker cannot go any further than "autovacuum-start-parallel-vacuum" point.
+# I.e. no one can interfere and change the number of free parallel workers.
+
+$psql_out = $node->safe_psql('postgres', qq{
+ SELECT get_parallel_autovacuum_free_workers();
+});
+is($psql_out, 10, 'All parallel workers has been released by the leader after FATAL');
+
+# Cleanup
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_detach('autovacuum-start-parallel-vacuum');
+ SELECT injection_points_detach('autovacuum-leader-before-indexes-processing');
+});
+
+$node->stop;
+done_testing();
diff --git a/src/test/modules/test_autovacuum/test_autovacuum--1.0.sql b/src/test/modules/test_autovacuum/test_autovacuum--1.0.sql
new file mode 100644
index 00000000000..e5646e0def5
--- /dev/null
+++ b/src/test/modules/test_autovacuum/test_autovacuum--1.0.sql
@@ -0,0 +1,12 @@
+/* src/test/modules/test_autovacuum/test_autovacuum--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION test_autovacuum" to load this file. \quit
+
+/*
+ * Functions for expecting shared autovacuum state
+ */
+
+CREATE FUNCTION get_parallel_autovacuum_free_workers()
+RETURNS INTEGER STRICT
+AS 'MODULE_PATHNAME' LANGUAGE C;
diff --git a/src/test/modules/test_autovacuum/test_autovacuum.c b/src/test/modules/test_autovacuum/test_autovacuum.c
new file mode 100644
index 00000000000..959629c7685
--- /dev/null
+++ b/src/test/modules/test_autovacuum/test_autovacuum.c
@@ -0,0 +1,41 @@
+/*-------------------------------------------------------------------------
+ *
+ * test_autovacuum.c
+ * Helpers to write tests for parallel autovacuum
+ *
+ * Copyright (c) 2020-2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/test/modules/test_autovacuum/test_autovacuum.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "commands/vacuum.h"
+#include "fmgr.h"
+#include "miscadmin.h"
+#include "postmaster/autovacuum.h"
+#include "storage/shmem.h"
+#include "storage/ipc.h"
+#include "storage/lwlock.h"
+#include "utils/builtins.h"
+#include "utils/injection_point.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(get_parallel_autovacuum_free_workers);
+Datum
+get_parallel_autovacuum_free_workers(PG_FUNCTION_ARGS)
+{
+ uint32 nfree_workers;
+
+#ifndef USE_INJECTION_POINTS
+ ereport(ERROR, errmsg("injection points not supported"));
+#endif
+
+ nfree_workers = AutoVacuumGetFreeParallelWorkers();
+
+ PG_RETURN_UINT32(nfree_workers);
+}
diff --git a/src/test/modules/test_autovacuum/test_autovacuum.control b/src/test/modules/test_autovacuum/test_autovacuum.control
new file mode 100644
index 00000000000..1b7fad258f0
--- /dev/null
+++ b/src/test/modules/test_autovacuum/test_autovacuum.control
@@ -0,0 +1,3 @@
+comment = 'Test code for parallel autovacuum'
+default_version = '1.0'
+module_pathname = '$libdir/test_autovacuum'
--
2.43.0
[text/x-patch] v21-0003-Cost-based-parameters-propagation-for-parallel-a.patch (9.8K, 6-v21-0003-Cost-based-parameters-propagation-for-parallel-a.patch)
download | inline diff:
From 21cbbbe37e36b53ac70b9827296a3430aba4680f Mon Sep 17 00:00:00 2001
From: Daniil Davidov <[email protected]>
Date: Thu, 15 Jan 2026 23:15:48 +0700
Subject: [PATCH v21 3/5] Cost based parameters propagation for parallel
autovacuum
---
src/backend/commands/vacuum.c | 23 +++-
src/backend/commands/vacuumparallel.c | 164 ++++++++++++++++++++++++++
src/backend/postmaster/autovacuum.c | 2 +-
src/include/commands/vacuum.h | 2 +
src/tools/pgindent/typedefs.list | 2 +
5 files changed, 190 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 03932f45c8a..70882544d05 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -2430,8 +2430,21 @@ vacuum_delay_point(bool is_analyze)
/* Always check for interrupts */
CHECK_FOR_INTERRUPTS();
- if (InterruptPending ||
- (!VacuumCostActive && !ConfigReloadPending))
+ if (InterruptPending)
+ return;
+
+ if (IsParallelWorker())
+ {
+ /*
+ * Possibly update cost-based delay parameters.
+ *
+ * Do it before checking VacuumCostActive, because its value might be
+ * changed after calling this function.
+ */
+ parallel_vacuum_update_shared_delay_params();
+ }
+
+ if (!VacuumCostActive && !ConfigReloadPending)
return;
/*
@@ -2445,6 +2458,12 @@ vacuum_delay_point(bool is_analyze)
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
VacuumUpdateCosts();
+
+ /*
+ * If we are parallel autovacuum leader and some of cost-based
+ * parameters had changed, let other parallel workers know.
+ */
+ parallel_vacuum_propagate_shared_delay_params();
}
/*
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 86d9f2b74c9..ccb3812165c 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -53,6 +53,56 @@
#define PARALLEL_VACUUM_KEY_WAL_USAGE 4
#define PARALLEL_VACUUM_KEY_INDEX_STATS 5
+/*
+ * Helper for the PVSharedCostParams structure (see below), to avoid
+ * repetition.
+ */
+typedef struct CostParamsData
+{
+ double cost_delay;
+ int cost_limit;
+ int cost_page_dirty;
+ int cost_page_hit;
+ int cost_page_miss;
+} CostParamsData;
+
+#define FillCostParamsData(cost_params) \
+ (cost_params)->cost_delay = vacuum_cost_delay; \
+ (cost_params)->cost_limit = vacuum_cost_limit; \
+ (cost_params)->cost_page_dirty = VacuumCostPageDirty; \
+ (cost_params)->cost_page_hit = VacuumCostPageHit; \
+ (cost_params)->cost_page_miss = VacuumCostPageMiss
+
+#define CostParamsDataEqual(params_1, params_2) \
+ ((params_1).cost_delay == (params_2).cost_delay && \
+ (params_1).cost_limit == (params_2).cost_limit && \
+ (params_1).cost_page_dirty == (params_2).cost_page_dirty && \
+ (params_1).cost_page_hit == (params_2).cost_page_hit && \
+ (params_1).cost_page_miss == (params_2).cost_page_miss)
+
+/*
+ * Struct for cost-based vacuum delay related parameters to share among an
+ * autovacuum worker and its parallel vacuum workers.
+ */
+typedef struct PVSharedCostParams
+{
+ /*
+ * Each time leader worker updates its parameters, it must increase
+ * generation. Every parallel worker keeps the generation
+ * (shared_params_local_generation) at which it had last time received
+ * parameters from the leader.
+ *
+ * It is enough for worker to compare it's local_generation with the field
+ * below to determine whether it needs to receive new parameters' values.
+ */
+ pg_atomic_uint32 generation;
+
+ slock_t mutex; /* protects all fields below */
+
+ /* Copies of corresponding parameters from autovacuum leader process */
+ CostParamsData params_data;
+} PVSharedCostParams;
+
/*
* Shared information among parallel workers. So this is allocated in the DSM
* segment.
@@ -122,6 +172,18 @@ typedef struct PVShared
/* Statistics of shared dead items */
VacDeadItemsInfo dead_items_info;
+
+ /*
+ * If 'true' then we are running parallel autovacuum. Otherwise, we are
+ * running parallel maintenence VACUUM.
+ */
+ bool am_parallel_autovacuum;
+
+ /*
+ * Struct for syncing parameters between supportive parallel autovacuum
+ * workers with leader worker.
+ */
+ PVSharedCostParams cost_params;
} PVShared;
/* Status used during parallel index vacuum or cleanup */
@@ -224,6 +286,11 @@ struct ParallelVacuumState
PVIndVacStatus status;
};
+static PVSharedCostParams *pv_shared_cost_params = NULL;
+
+/* See comments for the PVSharedCostParams structure for the explanation. */
+static uint32 shared_params_generation_local = 0;
+
static int parallel_vacuum_compute_workers(Relation *indrels, int nindexes, int nrequested,
bool *will_parallel_vacuum);
static void parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scans,
@@ -395,6 +462,17 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes,
pg_atomic_init_u32(&(shared->active_nworkers), 0);
pg_atomic_init_u32(&(shared->idx), 0);
+ shared->am_parallel_autovacuum = AmAutoVacuumWorkerProcess();
+
+ if (shared->am_parallel_autovacuum)
+ {
+ FillCostParamsData(&shared->cost_params.params_data);
+ pg_atomic_init_u32(&shared->cost_params.generation, 0);
+ SpinLockInit(&shared->cost_params.mutex);
+
+ pv_shared_cost_params = &(shared->cost_params);
+ }
+
shm_toc_insert(pcxt->toc, PARALLEL_VACUUM_KEY_SHARED, shared);
pvs->shared = shared;
@@ -539,6 +617,89 @@ parallel_vacuum_cleanup_all_indexes(ParallelVacuumState *pvs, long num_table_tup
&wusage->cleanup);
}
+/*
+ * If we are parallel *autovacuum* worker, check whether related to
+ * cost-based delay parameters had changed in the leader worker. If
+ * so, corresponding parameters will be updated to the values which
+ * leader worker is operating on.
+ *
+ * For non-autovacuum parallel worker this function will have no effect.
+ */
+void
+parallel_vacuum_update_shared_delay_params(void)
+{
+ uint32 params_generation;
+ CostParamsData shared_params_data;
+
+ Assert(IsParallelWorker());
+
+ /* Check whether we are running parallel autovacuum */
+ if (pv_shared_cost_params == NULL)
+ return;
+
+ params_generation = pg_atomic_read_u32(&pv_shared_cost_params->generation);
+ Assert(shared_params_generation_local <= params_generation);
+
+ /* Return if parameters had not changed in the leader */
+ if (params_generation == shared_params_generation_local)
+ return;
+
+ SpinLockAcquire(&pv_shared_cost_params->mutex);
+
+ shared_params_data = pv_shared_cost_params->params_data;
+
+ VacuumCostDelay = shared_params_data.cost_delay;
+ VacuumCostLimit = shared_params_data.cost_limit;
+ VacuumCostPageDirty = shared_params_data.cost_page_dirty;
+ VacuumCostPageHit = shared_params_data.cost_page_hit;
+ VacuumCostPageMiss = shared_params_data.cost_page_miss;
+
+ SpinLockRelease(&pv_shared_cost_params->mutex);
+
+ VacuumUpdateCosts();
+
+ shared_params_generation_local = params_generation;
+}
+
+/*
+ * Function to be called from parallel autovacuum leader in order to propagate
+ * some cost-based parameters to the supportive workers.
+ */
+void
+parallel_vacuum_propagate_shared_delay_params(void)
+{
+ CostParamsData local_params_data;
+
+ Assert(AmAutoVacuumWorkerProcess());
+
+ /* Check whether we are running parallel autovacuum */
+ if (pv_shared_cost_params == NULL)
+ return;
+
+ FillCostParamsData(&local_params_data);
+ SpinLockAcquire(&pv_shared_cost_params->mutex);
+
+ if (CostParamsDataEqual(pv_shared_cost_params->params_data,
+ local_params_data))
+ {
+ /*
+ * We don't need to update shared delay params if they haven't
+ * changed.
+ */
+ SpinLockRelease(&pv_shared_cost_params->mutex);
+ return;
+ }
+
+ FillCostParamsData(&pv_shared_cost_params->params_data);
+ SpinLockRelease(&pv_shared_cost_params->mutex);
+
+ /*
+ * Increase generation of the parameters, i.e. let parallel workers know
+ * that they should re-read shared cost params.
+ */
+ pg_atomic_fetch_add_u32(&pv_shared_cost_params->generation, 1);
+}
+
/*
* Compute the number of parallel worker processes to request. Both index
* vacuum and index cleanup can be executed with parallel workers.
@@ -1105,6 +1266,9 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
+ if (shared->am_parallel_autovacuum)
+ pv_shared_cost_params = &(shared->cost_params);
+
/* Set parallel vacuum state */
pvs.indrels = indrels;
pvs.nindexes = nindexes;
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index f40abe90ed5..0d78d02bd09 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -1690,7 +1690,7 @@ VacuumUpdateCosts(void)
}
else
{
- /* Must be explicit VACUUM or ANALYZE */
+ /* Must be explicit VACUUM or ANALYZE or parallel autovacuum worker */
vacuum_cost_delay = VacuumCostDelay;
vacuum_cost_limit = VacuumCostLimit;
}
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index d3dc4e8cc67..b10829a9379 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -423,6 +423,8 @@ extern void parallel_vacuum_cleanup_all_indexes(ParallelVacuumState *pvs,
int num_index_scans,
bool estimated_count,
PVWorkersUsage *wusage);
+extern void parallel_vacuum_update_shared_delay_params(void);
+extern void parallel_vacuum_propagate_shared_delay_params(void);
extern void parallel_vacuum_main(dsm_segment *seg, shm_toc *toc);
/* in commands/analyze.c */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index d84308c87ad..d00b57d3186 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -544,6 +544,7 @@ CopyToRoutine
CopyToState
CopyToStateData
Cost
+CostParamsData
CostSelector
Counters
CoverExt
@@ -2067,6 +2068,7 @@ PVIndStats
PVIndVacStatus
PVOID
PVShared
+PVSharedCostParams
PVWorkersUsage
PVWorkersStats
PX_Alias
--
2.43.0
[text/x-patch] v20--v21-diff-for-0003.patch (9.9K, 7-v20--v21-diff-for-0003.patch)
download | inline diff:
From 5e2b470db102546c3124da1eef8acc42d5c2fead Mon Sep 17 00:00:00 2001
From: Daniil Davidov <[email protected]>
Date: Sat, 7 Feb 2026 11:07:44 +0700
Subject: [PATCH 5/7] fixes for patch 3
---
src/backend/commands/vacuum.c | 14 +--
src/backend/commands/vacuumparallel.c | 120 ++++++++++++++++----------
src/include/commands/vacuum.h | 2 +-
src/tools/pgindent/typedefs.list | 2 +
4 files changed, 82 insertions(+), 56 deletions(-)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 9847ed1c2da..70882544d05 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -2433,19 +2433,13 @@ vacuum_delay_point(bool is_analyze)
if (InterruptPending)
return;
- if (!AmAutoVacuumWorkerProcess() && IsParallelWorker())
+ if (IsParallelWorker())
{
/*
- * If we are parallel *autovacuum* worker, check whether related to
- * cost-based delay parameters had changed in the leader worker. If
- * so, corresponding parameters will be updated to the values which
- * leader worker is operating on.
+ * Possibly update cost-based delay parameters.
*
* Do it before checking VacuumCostActive, because its value might be
- * changed after leader's parameters consumption.
- *
- * Note, that this function has no effect if we are non-autovacuum
- * parallel worker.
+ * changed after calling this function.
*/
parallel_vacuum_update_shared_delay_params();
}
@@ -2469,7 +2463,7 @@ vacuum_delay_point(bool is_analyze)
* If we are parallel autovacuum leader and some of cost-based
* parameters had changed, let other parallel workers know.
*/
- parallel_vacuum_propagate_cost_based_params();
+ parallel_vacuum_propagate_shared_delay_params();
}
/*
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 640173eada8..ccb3812165c 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -54,9 +54,35 @@
#define PARALLEL_VACUUM_KEY_INDEX_STATS 5
/*
- * Only autovacuum leader can reload config file. We use this structure in
- * parallel autovacuum for keeping worker's parameters in sync with leader's
- * parameters.
+ * Helper for the PVSharedCostParams structure (see below), to avoid
+ * repetition.
+ */
+typedef struct CostParamsData
+{
+ double cost_delay;
+ int cost_limit;
+ int cost_page_dirty;
+ int cost_page_hit;
+ int cost_page_miss;
+} CostParamsData;
+
+#define FillCostParamsData(cost_params) \
+ (cost_params)->cost_delay = vacuum_cost_delay; \
+ (cost_params)->cost_limit = vacuum_cost_limit; \
+ (cost_params)->cost_page_dirty = VacuumCostPageDirty; \
+ (cost_params)->cost_page_hit = VacuumCostPageHit; \
+ (cost_params)->cost_page_miss = VacuumCostPageMiss
+
+#define CostParamsDataEqual(params_1, params_2) \
+ ((params_1).cost_delay == (params_2).cost_delay && \
+ (params_1).cost_limit == (params_2).cost_limit && \
+ (params_1).cost_page_dirty == (params_2).cost_page_dirty && \
+ (params_1).cost_page_hit == (params_2).cost_page_hit && \
+ (params_1).cost_page_miss == (params_2).cost_page_miss)
+
+/*
+ * Struct for cost-based vacuum delay related parameters to share among an
+ * autovacuum worker and its parallel vacuum workers.
*/
typedef struct PVSharedCostParams
{
@@ -71,20 +97,11 @@ typedef struct PVSharedCostParams
*/
pg_atomic_uint32 generation;
- slock_t spinlock; /* protects all fields below */
+ slock_t mutex; /* protects all fields below */
/* Copies of corresponding parameters from autovacuum leader process */
- double cost_delay;
- int cost_limit;
- int cost_page_dirty;
- int cost_page_hit;
- int cost_page_miss;
-} PVSharedCostParams;
-
-static PVSharedCostParams * pv_shared_cost_params = NULL;
-
-/* See comments for structure above for the explanation. */
-static uint32 shared_params_generation_local = 0;
+ CostParamsData params_data;
+} PVSharedCostParams;
/*
* Shared information among parallel workers. So this is allocated in the DSM
@@ -269,6 +286,11 @@ struct ParallelVacuumState
PVIndVacStatus status;
};
+static PVSharedCostParams *pv_shared_cost_params = NULL;
+
+/* See comments for the PVSharedCostParams structure for the explanation. */
+static uint32 shared_params_generation_local = 0;
+
static int parallel_vacuum_compute_workers(Relation *indrels, int nindexes, int nrequested,
bool *will_parallel_vacuum);
static void parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scans,
@@ -444,13 +466,10 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes,
if (shared->am_parallel_autovacuum)
{
- shared->cost_params.cost_delay = vacuum_cost_delay;
- shared->cost_params.cost_limit = vacuum_cost_limit;
- shared->cost_params.cost_page_dirty = VacuumCostPageDirty;
- shared->cost_params.cost_page_hit = VacuumCostPageHit;
- shared->cost_params.cost_page_miss = VacuumCostPageMiss;
+ FillCostParamsData(&shared->cost_params.params_data);
pg_atomic_init_u32(&shared->cost_params.generation, 0);
- SpinLockInit(&shared->cost_params.spinlock);
+ SpinLockInit(&shared->cost_params.mutex);
+
pv_shared_cost_params = &(shared->cost_params);
}
@@ -599,13 +618,18 @@ parallel_vacuum_cleanup_all_indexes(ParallelVacuumState *pvs, long num_table_tup
}
/*
- * Function to be called from parallel autovacuum worker in order to sync
- * some cost-based delay parameter with the leader worker.
+ * If we are parallel *autovacuum* worker, check whether related to
+ * cost-based delay parameters had changed in the leader worker. If
+ * so, corresponding parameters will be updated to the values which
+ * leader worker is operating on.
+ *
+ * For non-autovacuum parallel worker this function will have no effect.
*/
void
parallel_vacuum_update_shared_delay_params(void)
{
- uint32 params_generation;
+ uint32 params_generation;
+ CostParamsData shared_params_data;
Assert(IsParallelWorker());
@@ -613,22 +637,24 @@ parallel_vacuum_update_shared_delay_params(void)
if (pv_shared_cost_params == NULL)
return;
- params_generation = pg_atomic_read_u32(&pv_shared_cost_params->generation);
+ params_generation = pg_atomic_read_u32(&pv_shared_cost_params->generation);
Assert(shared_params_generation_local <= params_generation);
/* Return if parameters had not changed in the leader */
if (params_generation == shared_params_generation_local)
return;
- SpinLockAcquire(&pv_shared_cost_params->spinlock);
+ SpinLockAcquire(&pv_shared_cost_params->mutex);
- VacuumCostDelay = pv_shared_cost_params->cost_delay;
- VacuumCostLimit = pv_shared_cost_params->cost_limit;
- VacuumCostPageDirty = pv_shared_cost_params->cost_page_dirty;
- VacuumCostPageHit = pv_shared_cost_params->cost_page_hit;
- VacuumCostPageMiss = pv_shared_cost_params->cost_page_miss;
+ shared_params_data = pv_shared_cost_params->params_data;
- SpinLockRelease(&pv_shared_cost_params->spinlock);
+ VacuumCostDelay = shared_params_data.cost_delay;
+ VacuumCostLimit = shared_params_data.cost_limit;
+ VacuumCostPageDirty = shared_params_data.cost_page_dirty;
+ VacuumCostPageHit = shared_params_data.cost_page_hit;
+ VacuumCostPageMiss = shared_params_data.cost_page_miss;
+
+ SpinLockRelease(&pv_shared_cost_params->mutex);
VacuumUpdateCosts();
@@ -640,9 +666,9 @@ parallel_vacuum_update_shared_delay_params(void)
* some cost-based parameters to the supportive workers.
*/
void
-parallel_vacuum_propagate_cost_based_params(void)
+parallel_vacuum_propagate_shared_delay_params(void)
{
- uint32 params_generation;
+ CostParamsData local_params_data;
Assert(AmAutoVacuumWorkerProcess());
@@ -650,24 +676,28 @@ parallel_vacuum_propagate_cost_based_params(void)
if (pv_shared_cost_params == NULL)
return;
- params_generation = pg_atomic_read_u32(&pv_shared_cost_params->generation);
+ FillCostParamsData(&local_params_data);
+ SpinLockAcquire(&pv_shared_cost_params->mutex);
- SpinLockAcquire(&pv_shared_cost_params->spinlock);
+ if (CostParamsDataEqual(pv_shared_cost_params->params_data,
+ local_params_data))
+ {
+ /*
+ * We don't need to update shared delay params if they haven't
+ * changed.
+ */
+ SpinLockRelease(&pv_shared_cost_params->mutex);
+ return;
+ }
- pv_shared_cost_params->cost_delay = vacuum_cost_delay;
- pv_shared_cost_params->cost_limit = vacuum_cost_limit;
- pv_shared_cost_params->cost_page_dirty = VacuumCostPageDirty;
- pv_shared_cost_params->cost_page_hit = VacuumCostPageHit;
- pv_shared_cost_params->cost_page_miss = VacuumCostPageMiss;
+ FillCostParamsData(&pv_shared_cost_params->params_data);
+ SpinLockRelease(&pv_shared_cost_params->mutex);
/*
* Increase generation of the parameters, i.e. let parallel workers know
* that they should re-read shared cost params.
*/
- pg_atomic_write_u32(&pv_shared_cost_params->generation,
- params_generation + 1);
-
- SpinLockRelease(&pv_shared_cost_params->spinlock);
+ pg_atomic_fetch_add_u32(&pv_shared_cost_params->generation, 1);
}
/*
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index bd67e7748f0..b10829a9379 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -424,7 +424,7 @@ extern void parallel_vacuum_cleanup_all_indexes(ParallelVacuumState *pvs,
bool estimated_count,
PVWorkersUsage *wusage);
extern void parallel_vacuum_update_shared_delay_params(void);
-extern void parallel_vacuum_propagate_cost_based_params(void);
+extern void parallel_vacuum_propagate_shared_delay_params(void);
extern void parallel_vacuum_main(dsm_segment *seg, shm_toc *toc);
/* in commands/analyze.c */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 84bfa2970de..28b91d69086 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -544,6 +544,7 @@ CopyToRoutine
CopyToState
CopyToStateData
Cost
+CostParamsData
CostSelector
Counters
CoverExt
@@ -2066,6 +2067,7 @@ PVIndStats
PVIndVacStatus
PVOID
PVShared
+PVSharedCostParams
PVWorkersUsage
PVWorkersStats
PX_Alias
--
2.43.0
[text/x-patch] v20--v21-diff-for-0001.patch (3.5K, 8-v20--v21-diff-for-0001.patch)
download | inline diff:
From cb6f66f944dbb48a31c90823dd23a6a5d6313250 Mon Sep 17 00:00:00 2001
From: Daniil Davidov <[email protected]>
Date: Sat, 7 Feb 2026 00:04:17 +0700
Subject: [PATCH 1/7] fixes for patch 1
---
src/backend/commands/vacuumparallel.c | 5 ++++-
src/backend/postmaster/autovacuum.c | 18 +++++-------------
src/backend/utils/misc/postgresql.conf.sample | 2 +-
3 files changed, 10 insertions(+), 15 deletions(-)
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 1e35b82aeaf..d3e0c32b7ee 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -710,8 +710,11 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
* Tell autovacuum that we could not launch all the previously
* reserved workers.
*/
- if (AmAutoVacuumWorkerProcess() && pvs->pcxt->nworkers_launched < nworkers)
+ if (AmAutoVacuumWorkerProcess() &&
+ pvs->pcxt->nworkers_launched < nworkers)
+ {
AutoVacuumReleaseParallelWorkers(nworkers - pvs->pcxt->nworkers_launched);
+ }
if (pvs->pcxt->nworkers_launched > 0)
{
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 784c1178d61..f40abe90ed5 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -3526,7 +3526,7 @@ AutoVacuumShmemInit(void)
AutoVacuumShmem->av_launcherpid = 0;
AutoVacuumShmem->av_maxParallelWorkers =
- Min(autovacuum_max_parallel_workers, max_worker_processes);
+ Min(autovacuum_max_parallel_workers, max_parallel_workers);
AutoVacuumShmem->av_freeParallelWorkers =
AutoVacuumShmem->av_maxParallelWorkers;
dclist_init(&AutoVacuumShmem->av_freeWorkers);
@@ -3622,23 +3622,15 @@ adjust_free_parallel_workers(int prev_max_parallel_workers)
LWLockAcquire(AutovacuumLock, LW_EXCLUSIVE);
- nfree_workers =
- autovacuum_max_parallel_workers - prev_max_parallel_workers +
- AutoVacuumShmem->av_freeParallelWorkers;
-
/*
* Cap or increase number of free parallel workers according to the
* parameter change.
*/
- AutoVacuumShmem->av_freeParallelWorkers = Max(nfree_workers, 0);
-
- /*
- * Don't allow number of free workers to become less than zero if the
- * patameter was decreased.
- */
- AutoVacuumShmem->av_freeParallelWorkers =
- Max(AutoVacuumShmem->av_freeParallelWorkers, 0);
+ nfree_workers =
+ autovacuum_max_parallel_workers - prev_max_parallel_workers +
+ AutoVacuumShmem->av_freeParallelWorkers;
+ AutoVacuumShmem->av_freeParallelWorkers = Max(nfree_workers, 0);
AutoVacuumShmem->av_maxParallelWorkers = autovacuum_max_parallel_workers;
LWLockRelease(AutovacuumLock);
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 35c37f21239..e456fd759eb 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -695,7 +695,7 @@
#autovacuum_worker_slots = 16 # autovacuum worker slots to allocate
# (change requires restart)
#autovacuum_max_workers = 3 # max number of autovacuum subprocesses
-#autovacuum_max_parallel_workers = 2 # limited by max_worker_processes
+#autovacuum_max_parallel_workers = 2 # limited by max_parallel_workers
#autovacuum_naptime = 1min # time between autovacuum runs
#autovacuum_vacuum_threshold = 50 # min number of row updates before
# vacuum
--
2.43.0
[text/x-patch] v20--v21-diff-for-0002.patch (8.3K, 9-v20--v21-diff-for-0002.patch)
download | inline diff:
From b7a0226646ee306400ca50c5404f3f02b0c7fda0 Mon Sep 17 00:00:00 2001
From: Daniil Davidov <[email protected]>
Date: Sat, 7 Feb 2026 00:11:49 +0700
Subject: [PATCH 3/7] fixes for patch 2
---
src/backend/access/heap/vacuumlazy.c | 60 ++++++++++++++++++++-------
src/backend/commands/vacuumparallel.c | 22 +++++-----
src/include/commands/vacuum.h | 15 +++++--
src/tools/pgindent/typedefs.list | 2 +
4 files changed, 70 insertions(+), 29 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d14f055b40d..d19e15cbcce 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -784,8 +784,10 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
vacrel->vm_new_visible_frozen_pages = 0;
vacrel->vm_new_frozen_pages = 0;
- vacrel->workers_usage.nlaunched = 0;
- vacrel->workers_usage.nplanned = 0;
+ vacrel->workers_usage.vacuum.nlaunched = 0;
+ vacrel->workers_usage.vacuum.nplanned = 0;
+ vacrel->workers_usage.cleanup.nlaunched = 0;
+ vacrel->workers_usage.cleanup.nplanned = 0;
/*
* Get cutoffs that determine which deleted tuples are considered DEAD,
@@ -1129,23 +1131,49 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
orig_rel_pages == 0 ? 100.0 :
100.0 * vacrel->lpdead_item_pages / orig_rel_pages,
vacrel->lpdead_items);
- if (vacrel->workers_usage.nplanned > 0 &&
- AmAutoVacuumWorkerProcess())
+ if (vacrel->workers_usage.vacuum.nplanned > 0)
{
- /* Worker usage stats for parallel autovacuum */
- appendStringInfo(&buf,
- _("parallel index vacuum/cleanup: %d workers were planned, %d workers were reserved and %d workers were launched in total\n"),
- vacrel->workers_usage.nplanned,
- vacrel->workers_usage.nreserved,
- vacrel->workers_usage.nlaunched);
+ /* Stats for vacuum phase of index vacuuming. */
+
+ if (AmAutoVacuumWorkerProcess())
+ {
+ /* Worker usage stats for parallel autovacuum. */
+ appendStringInfo(&buf,
+ _("parallel index vacuum: %d workers were planned, %d workers were reserved and %d workers were launched in total\n"),
+ vacrel->workers_usage.vacuum.nplanned,
+ vacrel->workers_usage.vacuum.nreserved,
+ vacrel->workers_usage.vacuum.nlaunched);
+ }
+ else
+ {
+ /* Worker usage stats for manual VACUUM (PARALLEL). */
+ appendStringInfo(&buf,
+ _("parallel index vacuum: %d workers were planned and %d workers were launched in total\n"),
+ vacrel->workers_usage.vacuum.nplanned,
+ vacrel->workers_usage.vacuum.nlaunched);
+ }
}
- else if (vacrel->workers_usage.nplanned > 0)
+ if (vacrel->workers_usage.cleanup.nplanned > 0)
{
- /* Worker usage stats for manual VACUUM (PARALLEL) */
- appendStringInfo(&buf,
- _("parallel index vacuum/cleanup: %d workers were planned and %d workers were launched in total\n"),
- vacrel->workers_usage.nplanned,
- vacrel->workers_usage.nlaunched);
+ /* Stats for cleanup phase of index vacuuming. */
+
+ if (AmAutoVacuumWorkerProcess())
+ {
+ /* Worker usage stats for parallel autovacuum. */
+ appendStringInfo(&buf,
+ _("parallel index cleanup: %d workers were planned, %d workers were reserved and %d workers were launched in total\n"),
+ vacrel->workers_usage.cleanup.nplanned,
+ vacrel->workers_usage.cleanup.nreserved,
+ vacrel->workers_usage.cleanup.nlaunched);
+ }
+ else
+ {
+ /* Worker usage stats for manual VACUUM (PARALLEL). */
+ appendStringInfo(&buf,
+ _("parallel index cleanup: %d workers were planned and %d workers were launched in total\n"),
+ vacrel->workers_usage.cleanup.nplanned,
+ vacrel->workers_usage.cleanup.nlaunched);
+ }
}
for (int i = 0; i < vacrel->nindexes; i++)
{
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index ea45dc3fc37..86d9f2b74c9 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -227,7 +227,7 @@ struct ParallelVacuumState
static int parallel_vacuum_compute_workers(Relation *indrels, int nindexes, int nrequested,
bool *will_parallel_vacuum);
static void parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scans,
- bool vacuum, PVWorkersUsage *wusage);
+ bool vacuum, PVWorkersStats *wstats);
static void parallel_vacuum_process_safe_indexes(ParallelVacuumState *pvs);
static void parallel_vacuum_process_unsafe_indexes(ParallelVacuumState *pvs);
static void parallel_vacuum_process_one_index(ParallelVacuumState *pvs, Relation indrel,
@@ -513,7 +513,8 @@ parallel_vacuum_bulkdel_all_indexes(ParallelVacuumState *pvs, long num_table_tup
pvs->shared->reltuples = num_table_tuples;
pvs->shared->estimated_count = true;
- parallel_vacuum_process_all_indexes(pvs, num_index_scans, true, wusage);
+ parallel_vacuum_process_all_indexes(pvs, num_index_scans, true,
+ &wusage->vacuum);
}
/*
@@ -534,7 +535,8 @@ parallel_vacuum_cleanup_all_indexes(ParallelVacuumState *pvs, long num_table_tup
pvs->shared->reltuples = num_table_tuples;
pvs->shared->estimated_count = estimated_count;
- parallel_vacuum_process_all_indexes(pvs, num_index_scans, false, wusage);
+ parallel_vacuum_process_all_indexes(pvs, num_index_scans, false,
+ &wusage->cleanup);
}
/*
@@ -619,7 +621,7 @@ parallel_vacuum_compute_workers(Relation *indrels, int nindexes, int nrequested,
*/
static void
parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scans,
- bool vacuum, PVWorkersUsage *wusage)
+ bool vacuum, PVWorkersStats *wstats)
{
int nworkers;
PVIndVacStatus new_status;
@@ -657,8 +659,8 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
nworkers = Min(nworkers, pvs->pcxt->nworkers);
/* Remember this value, if we asked to */
- if (wusage != NULL && nworkers > 0)
- wusage->nplanned += nworkers;
+ if (wstats != NULL && nworkers > 0)
+ wstats->nplanned += nworkers;
/*
* Reserve workers in autovacuum global state. Note that we may be given
@@ -669,8 +671,8 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
AutoVacuumReserveParallelWorkers(&nworkers);
/* Remember this value, if we asked to */
- if (wusage != NULL)
- wusage->nreserved += nworkers;
+ if (wstats != NULL)
+ wstats->nreserved += nworkers;
}
/*
@@ -741,8 +743,8 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
VacuumActiveNWorkers = &(pvs->shared->active_nworkers);
/* Remember this value, if we asked to */
- if (wusage != NULL)
- wusage->nlaunched += pvs->pcxt->nworkers_launched;
+ if (wstats != NULL)
+ wstats->nlaunched += pvs->pcxt->nworkers_launched;
}
if (vacuum)
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 7cbb59d124f..d3dc4e8cc67 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -301,16 +301,25 @@ typedef struct VacDeadItemsInfo
} VacDeadItemsInfo;
/*
- * PVWorkersUsage stores information about total number of launched, reserved
- * and planned workers during parallel vacuum.
+ * Helper for the PVWorkersUsage structure (see below), to avoid repetition.
*/
-typedef struct PVWorkersUsage
+typedef struct PVWorkersStats
{
int nplanned; /* # of parallel workers we are planned to
* launch */
int nreserved; /* for autovacuum only - # of parallel workers
* we have managed to reserve */
int nlaunched; /* # of launched parallel workers */
+} PVWorkersStats;
+
+/*
+ * PVWorkersUsage stores information about total number of launched, reserved
+ * and planned workers during parallel vacuum (both for vacuum and cleanup).
+ */
+typedef struct PVWorkersUsage
+{
+ PVWorkersStats vacuum;
+ PVWorkersStats cleanup;
} PVWorkersUsage;
/* GUC parameters */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1988cd874fd..84bfa2970de 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2066,6 +2066,8 @@ PVIndStats
PVIndVacStatus
PVOID
PVShared
+PVWorkersUsage
+PVWorkersStats
PX_Alias
PX_Cipher
PX_Combo
--
2.43.0
[text/x-patch] v20--v21-diff-for-0004.patch (18.3K, 10-v20--v21-diff-for-0004.patch)
download | inline diff:
From 82c1f442ba5e18e3d6dc3cb4ed4f24cd3a8d910f Mon Sep 17 00:00:00 2001
From: Daniil Davidov <[email protected]>
Date: Tue, 10 Feb 2026 21:31:14 +0700
Subject: [PATCH 7/7] fixes for patch 4
---
src/backend/access/heap/vacuumlazy.c | 7 +
src/backend/commands/vacuumparallel.c | 32 ++-
src/backend/postmaster/autovacuum.c | 19 +-
.../modules/test_autovacuum/t/001_basic.pl | 188 ++++++++----------
4 files changed, 121 insertions(+), 125 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d19e15cbcce..2e85f7f17f7 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -151,6 +151,7 @@
#include "storage/freespace.h"
#include "storage/lmgr.h"
#include "storage/read_stream.h"
+#include "utils/injection_point.h"
#include "utils/lsyscache.h"
#include "utils/pg_rusage.h"
#include "utils/timestamp.h"
@@ -869,6 +870,12 @@ heap_vacuum_rel(Relation rel, const VacuumParams params,
lazy_check_wraparound_failsafe(vacrel);
dead_items_alloc(vacrel, params.nworkers);
+ /*
+ * Trigger injection point, if parallel autovacuum is about to be started.
+ */
+ if (AmAutoVacuumWorkerProcess() && ParallelVacuumIsActive(vacrel))
+ INJECTION_POINT("autovacuum-start-parallel-vacuum", NULL);
+
/*
* Call lazy_scan_heap to perform all required heap pruning, index
* vacuuming, and heap vacuuming (plus related processing)
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 13649747322..5dad19d8ed8 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -928,6 +928,9 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
* To be able to exercise whether all reserved parallel workers are being
* released anyway, allow injection points to trigger a failure at this
* point.
+ *
+ * This injection point is also used to wait until parallel workers
+ * finishes their part of index processing.
*/
if (nworkers > 0)
INJECTION_POINT("autovacuum-leader-before-indexes-processing", NULL);
@@ -941,15 +944,6 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
*/
parallel_vacuum_process_safe_indexes(pvs);
- /*
- * To be able to exercise whether leader parallel autovacuum worker can
- * propagate cost-based params to parallel workers, wait here until
- * configuration is changed. I.e. tests are expecting, that during index
- * processing vacuum_delay_point have been called (if config was changed).
- */
- if (nworkers > 0)
- INJECTION_POINT("autovacuum-leader-after-indexes-processing", NULL);
-
/*
* Next, accumulate buffer and WAL usage. (This must wait for the workers
* to finish, or we might get incomplete data.)
@@ -1315,20 +1309,16 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Prepare to track buffer usage during parallel execution */
InstrStartParallelQuery();
- INJECTION_POINT("parallel-worker-before-indexes-processing", NULL);
-
/* Process indexes to perform vacuum/cleanup */
parallel_vacuum_process_safe_indexes(&pvs);
#ifdef USE_INJECTION_POINTS
/*
- * There is no guarantee that each parallel worker will necessarily
- * process at least one index. Thus, at this point we cannot be sure that
- * worker called vacuum_cost_delay. In order to test cost-based parameters
- * propagation (from leader worker), call vacuum_delay_point here, if
- * injection point is active.
+ * If we are parallel autovacuum worker, we can consume delay parameters
+ * during index processing (via vacuum_delay_point call). This logging
+ * allows tests to ensure this.
*/
- if (IS_INJECTION_POINT_ATTACHED("parallel-autovacuum-force-delay-point"))
+ if (shared->am_parallel_autovacuum)
parallel_vacuum_report_cost_based_params();
#endif
@@ -1392,6 +1382,7 @@ parallel_vacuum_error_callback(void *arg)
static void
parallel_vacuum_report_cost_based_params(void)
{
+#ifdef USE_INJECTION_POINTS
StringInfoData buf;
/* Simulate config reload during normal processing */
@@ -1402,12 +1393,15 @@ parallel_vacuum_report_cost_based_params(void)
initStringInfo(&buf);
appendStringInfo(&buf, "Vacuum cost-based delay parameters of parallel worker:\n");
- appendStringInfo(&buf,"vacuum_cost_limit = %d\n",vacuum_cost_limit);
+ appendStringInfo(&buf, "vacuum_cost_limit = %d\n",vacuum_cost_limit);
appendStringInfo(&buf, "vacuum_cost_delay = %g\n", vacuum_cost_delay);
appendStringInfo(&buf, "vacuum_cost_page_miss = %d\n", VacuumCostPageMiss);
appendStringInfo(&buf, "vacuum_cost_page_dirty = %d\n", VacuumCostPageDirty);
appendStringInfo(&buf, "vacuum_cost_page_hit = %d\n", VacuumCostPageHit);
- ereport(LOG, errmsg("%s", buf.data));
+ ereport(DEBUG2, errmsg("%s", buf.data));
pfree(buf.data);
+#else
+ elog(ERROR, "Injection points are not supported by this build");
+#endif
}
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index b9ff60be0f2..7b24a5d6e67 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -800,8 +800,6 @@ ProcessAutoVacLauncherInterrupts(void)
/* rebuild the list in case the naptime changed */
rebuild_database_list(InvalidOid);
-
- INJECTION_POINT("autovacuum-launcher-after-reload-config", NULL);
}
/* Process barrier events */
@@ -2497,12 +2495,20 @@ do_autovacuum(void)
}
PG_CATCH();
{
+ int nreserved_workers = av_nworkers_reserved;
+
/*
* Parallel autovacuum can reserve parallel workers. Make sure
* that all reserved workers are released.
*/
AutoVacuumReleaseAllParallelWorkers();
+ if (nreserved_workers > 0)
+ ereport(DEBUG2,
+ (errmsg("%d parallel autovacuum workers has been released after occured error",
+ nreserved_workers),
+ errhidecontext(true)));
+
/*
* Abort the transaction, start a new one, and proceed with the
* next table in our list.
@@ -3469,15 +3475,13 @@ AutoVacuumReleaseAllParallelWorkers(void)
/*
* Get number of free autovacuum parallel workers.
- *
- * For testing purpose only!
*/
uint32
AutoVacuumGetFreeParallelWorkers(void)
{
uint32 nfree_workers;
- LWLockAcquire(AutovacuumLock, LW_EXCLUSIVE);
+ LWLockAcquire(AutovacuumLock, LW_SHARED);
nfree_workers = AutoVacuumShmem->av_freeParallelWorkers;
LWLockRelease(AutovacuumLock);
@@ -3652,5 +3656,10 @@ adjust_free_parallel_workers(int prev_max_parallel_workers)
AutoVacuumShmem->av_freeParallelWorkers = Max(nfree_workers, 0);
AutoVacuumShmem->av_maxParallelWorkers = autovacuum_max_parallel_workers;
+ ereport(DEBUG2,
+ (errmsg("number of free parallel autovacuum workers is set to %u due to config reload",
+ AutoVacuumShmem->av_freeParallelWorkers),
+ errhidecontext(true)));
+
LWLockRelease(AutovacuumLock);
}
diff --git a/src/test/modules/test_autovacuum/t/001_basic.pl b/src/test/modules/test_autovacuum/t/001_basic.pl
index 065a58ef2e6..c5d8fffc47c 100644
--- a/src/test/modules/test_autovacuum/t/001_basic.pl
+++ b/src/test/modules/test_autovacuum/t/001_basic.pl
@@ -1,3 +1,5 @@
+# Test parallel autovacuum behavior
+
use warnings FATAL => 'all';
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
@@ -21,17 +23,9 @@ sub prepare_for_next_test
$node->safe_psql('postgres', qq{
UPDATE test_autovac SET col_1 = $test_number;
- ANALYZE test_autovac;
});
}
-sub wait_for_av_log
-{
- my ($node, $expected_log) = @_;
-
- $node->wait_for_log($expected_log);
- truncate $node->logfile, 0 or die "truncate failed: $!";
-}
my $psql_out;
@@ -71,31 +65,30 @@ my $indexes_num = 4;
my $initial_rows_num = 10_000;
my $autovacuum_parallel_workers = 2;
-# Create table with specified number of b-tree indexes on it
+# Create table and fill it with some data
$node->safe_psql('postgres', qq{
CREATE TABLE test_autovac (
id SERIAL PRIMARY KEY,
col_1 INTEGER, col_2 INTEGER, col_3 INTEGER, col_4 INTEGER
) WITH (autovacuum_parallel_workers = $autovacuum_parallel_workers);
- DO \$\$
- DECLARE
- i INTEGER;
- BEGIN
- FOR i IN 1..$indexes_num LOOP
- EXECUTE format('CREATE INDEX idx_col_\%s ON test_autovac (col_\%s);', i, i);
- END LOOP;
- END \$\$;
+ INSERT INTO test_autovac
+ SELECT
+ g AS col1,
+ g + 1 AS col2,
+ g + 2 AS col3,
+ g + 3 AS col4
+ FROM generate_series(1, $initial_rows_num) AS g;
});
-# Insert specified tuples num into the table
+# Create specified number of b-tree indexes on the table
$node->safe_psql('postgres', qq{
DO \$\$
DECLARE
i INTEGER;
BEGIN
- FOR i IN 1..$initial_rows_num LOOP
- INSERT INTO test_autovac VALUES (i, i + 1, i + 2, i + 3);
+ FOR i IN 1..$indexes_num LOOP
+ EXECUTE format('CREATE INDEX idx_col_\%s ON test_autovac (col_\%s);', i, i);
END LOOP;
END \$\$;
});
@@ -115,14 +108,15 @@ $node->safe_psql('postgres', qq{
# Wait until the parallel autovacuum on table is completed. At the same time,
# we check that the required number of parallel workers has been started.
-wait_for_av_log($node,
- qr/parallel index vacuum\/cleanup: 2 workers were planned, / .
- qr/2 workers were reserved and 2 workers were launched in total/);
-
-$node->psql('postgres',
- "SELECT get_parallel_autovacuum_free_workers();",
- stdout => \$psql_out,
+$log_start = $node->wait_for_log(
+ qr/parallel index vacuum: 2 workers were planned, / .
+ qr/2 workers were reserved and 2 workers were launched in total/,
+ $log_start
);
+
+$psql_out = $node->safe_psql('postgres', qq{
+ SELECT get_parallel_autovacuum_free_workers();
+});
is($psql_out, 20, 'All parallel workers has been released by the leader');
# Test 2:
@@ -132,19 +126,16 @@ is($psql_out, 20, 'All parallel workers has been released by the leader');
prepare_for_next_test($node, 2);
$node->safe_psql('postgres', qq{
+ SELECT injection_points_attach('autovacuum-start-parallel-vacuum', 'wait');
SELECT injection_points_attach('autovacuum-leader-before-indexes-processing', 'wait');
- SELECT injection_points_attach('autovacuum-leader-after-indexes-processing', 'wait');
- SELECT injection_points_attach('parallel-worker-before-indexes-processing', 'wait');
- SELECT injection_points_attach('parallel-autovacuum-force-delay-point', 'wait');
ALTER TABLE test_autovac SET (autovacuum_parallel_workers = 1, autovacuum_enabled = true);
});
-# Wait until parallel autovacuum leader launches parallel worker and falls
-# asleep on the injection point
+# Wait until parallel autovacuum is inited
$node->wait_for_event(
'autovacuum worker',
- 'autovacuum-leader-before-indexes-processing'
+ 'autovacuum-start-parallel-vacuum'
);
# Reload config - leader worker must update its own parameters during indexes
@@ -158,45 +149,34 @@ $node->safe_psql('postgres', qq{
});
$node->safe_psql('postgres', qq{
- SELECT injection_points_wakeup('autovacuum-leader-before-indexes-processing');
+ SELECT injection_points_wakeup('autovacuum-start-parallel-vacuum');
});
-# Wait until leader worker is guaranteed to update parameters and propagate
-# their values to the parallel worker
+# Now wait until parallel autovacuum leader completes processing table (i.e.
+# guaranteed to call vacuum_delay_point) and launches parallel worker.
$node->wait_for_event(
'autovacuum worker',
- 'autovacuum-leader-after-indexes-processing'
+ 'autovacuum-leader-before-indexes-processing'
);
-$node->safe_psql('postgres', qq{
- SELECT injection_points_wakeup('autovacuum-leader-after-indexes-processing');
-});
-
-# Now wake up the parallel worker and force it to call vacuum_delay_point
-$node->wait_for_event(
- 'parallel worker',
- 'parallel-worker-before-indexes-processing'
+# Check whether parallel worker successfully updated all parameters during
+# index processing
+$log_start = $node->wait_for_log(
+ qr/Vacuum cost-based delay parameters of parallel worker:\n/ .
+ qr/\tvacuum_cost_limit = 500\n/ .
+ qr/\tvacuum_cost_delay = 2\n/ .
+ qr/\tvacuum_cost_page_miss = 10\n/ .
+ qr/\tvacuum_cost_page_dirty = 10\n/ .
+ qr/\tvacuum_cost_page_hit = 10\n/,
+ $log_start
);
-$node->safe_psql('postgres', qq{
- SELECT injection_points_wakeup('parallel-worker-before-indexes-processing');
-});
-
-# Check whether worker successfully updated all parameters
-wait_for_av_log($node,
- qr/Vacuum cost-based delay parameters of parallel worker:\n/ .
- qr/\tvacuum_cost_limit = 500\n/ .
- qr/\tvacuum_cost_delay = 2\n/ .
- qr/\tvacuum_cost_page_miss = 10\n/ .
- qr/\tvacuum_cost_page_dirty = 10\n/ .
- qr/\tvacuum_cost_page_hit = 10\n/);
-
# Cleanup
$node->safe_psql('postgres', qq{
+ SELECT injection_points_wakeup('autovacuum-leader-before-indexes-processing');
+
+ SELECT injection_points_detach('autovacuum-start-parallel-vacuum');
SELECT injection_points_detach('autovacuum-leader-before-indexes-processing');
- SELECT injection_points_detach('autovacuum-leader-after-indexes-processing');
- SELECT injection_points_detach('parallel-worker-before-indexes-processing');
- SELECT injection_points_detach('parallel-autovacuum-force-delay-point');
ALTER TABLE test_autovac SET (autovacuum_parallel_workers = $autovacuum_parallel_workers);
});
@@ -209,7 +189,6 @@ prepare_for_next_test($node, 4);
$node->safe_psql('postgres', qq{
SELECT injection_points_attach('autovacuum-leader-before-indexes-processing', 'wait');
- SELECT injection_points_attach('autovacuum-launcher-after-reload-config', 'wait');
ALTER TABLE test_autovac SET (autovacuum_enabled = true);
});
@@ -223,53 +202,44 @@ $node->safe_psql('postgres', qq{
SELECT pg_reload_conf();
});
-$node->wait_for_event(
- 'autovacuum launcher',
- 'autovacuum-launcher-after-reload-config'
-);
-
# Since 2 parallel workers already launched and will be released in the future,
# we are expecting that :
# 1) number of free workers will be '0' after config reload
# 2) number of free workers will be '1' after releasing workers
# Check statement (1)
-$node->psql('postgres',
- "SELECT get_parallel_autovacuum_free_workers();",
- stdout => \$psql_out,
+$log_start = $node->wait_for_log(
+ qr/number of free parallel autovacuum workers is set to 0 due to config reload/,
+ $log_start
);
-is($psql_out, 0,
- 'Number of free parallel workers is consistent');
$node->safe_psql('postgres', qq{
- SELECT injection_points_wakeup('autovacuum-launcher-after-reload-config');
SELECT injection_points_wakeup('autovacuum-leader-before-indexes-processing');
});
# Wait until the end of parallel processing
-wait_for_av_log($node,
- qr/parallel index vacuum\/cleanup: 2 workers were planned, / .
- qr/2 workers were reserved and 2 workers were launched in total/);
+$log_start = $node->wait_for_log(
+ qr/parallel index vacuum: 2 workers were planned, / .
+ qr/2 workers were reserved and 2 workers were launched in total/,
+ $log_start
+);
# Check statement (2)
-$node->psql('postgres',
- "SELECT get_parallel_autovacuum_free_workers();",
- stdout => \$psql_out,
-);
-is($psql_out, 1,
- 'Number of free parallel workers is consistent');
+$psql_out = $node->safe_psql('postgres', qq{
+ SELECT get_parallel_autovacuum_free_workers();
+});
+is($psql_out, 1, 'Number of free parallel workers is consistent');
# Cleanup
$node->safe_psql('postgres', qq{
SELECT injection_points_detach('autovacuum-leader-before-indexes-processing');
- SELECT injection_points_detach('autovacuum-launcher-after-reload-config');
ALTER SYSTEM SET autovacuum_max_parallel_workers = 10;
SELECT pg_reload_conf();
});
# Test 4:
# We want parallel autovacuum workers to be released even if leader gets an
-# error. At first, simulate situation, when leader exites due to an ERROR.
+# error. At first, simulate situation, when leader exits due to an ERROR.
prepare_for_next_test($node, 4);
@@ -278,16 +248,16 @@ $node->safe_psql('postgres', qq{
ALTER TABLE test_autovac SET (autovacuum_enabled = true);
});
-wait_for_av_log($node,
- qr/error triggered for injection point / .
- qr/autovacuum-leader-before-indexes-processing/);
+$log_start = $node->wait_for_log(
+ qr/error triggered for injection point / .
+ qr/autovacuum-leader-before-indexes-processing/,
+ $log_start
+);
-$node->psql('postgres',
- "SELECT get_parallel_autovacuum_free_workers();",
- stdout => \$psql_out,
+$log_start = $node->wait_for_log(
+ qr/2 parallel autovacuum workers has been released after occured error/,
+ $log_start
);
-is($psql_out, 10,
- 'All parallel workers has been released by the leader after ERROR');
# Cleanup
$node->safe_psql('postgres', qq{
@@ -295,15 +265,25 @@ $node->safe_psql('postgres', qq{
});
# Test 5:
-# Same as above test, but simulate situation, when leader exites due to FATAL.
+# Same as above test, but simulate situation, when leader exits due to FATAL.
prepare_for_next_test($node, 5);
$node->safe_psql('postgres', qq{
+ SELECT injection_points_attach('autovacuum-start-parallel-vacuum', 'wait');
SELECT injection_points_attach('autovacuum-leader-before-indexes-processing', 'wait');
ALTER TABLE test_autovac SET (autovacuum_enabled = true);
});
+# Wait until parallel autovacuum is inited and wake up the leader
+$node->wait_for_event(
+ 'autovacuum worker',
+ 'autovacuum-start-parallel-vacuum'
+);
+$node->safe_psql('postgres', qq{
+ SELECT injection_points_wakeup('autovacuum-start-parallel-vacuum');
+});
+
$node->wait_for_event(
'autovacuum worker',
'autovacuum-leader-before-indexes-processing'
@@ -327,18 +307,24 @@ $node->safe_psql('postgres', qq{
SELECT pg_terminate_backend('$av_pid');
});
-wait_for_av_log($node,
- qr/terminating autovacuum process due to administrator command/);
-
-$node->psql('postgres',
- "SELECT get_parallel_autovacuum_free_workers();",
- stdout => \$psql_out,
+$log_start = $node->wait_for_log(
+ qr/terminating autovacuum process due to administrator command/,
+ $log_start
);
-is($psql_out, 10,
- 'All parallel workers has been released by the leader after FATAL');
+
+# Now it is safe to check the number of free parallel workers, because even if
+# autovacuum is trying to vacuum table in parallel mode again, the leader
+# worker cannot go any further than "autovacuum-start-parallel-vacuum" point.
+# I.e. no one can interfere and change the number of free parallel workers.
+
+$psql_out = $node->safe_psql('postgres', qq{
+ SELECT get_parallel_autovacuum_free_workers();
+});
+is($psql_out, 10, 'All parallel workers has been released by the leader after FATAL');
# Cleanup
$node->safe_psql('postgres', qq{
+ SELECT injection_points_detach('autovacuum-start-parallel-vacuum');
SELECT injection_points_detach('autovacuum-leader-before-indexes-processing');
});
--
2.43.0
view thread (112+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: POC: Parallel processing of indexes in autovacuum
In-Reply-To: <CAJDiXggY1QzNde6_HhpzneLc9dYqmWZ+PY39cuBXYdcCTuoJBA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox