MIME-Version: 1.0
References: 
 <CACG=ezZOrNsuLoETLD1gAswZMuH2nGGq7Ogcc0QOE5hhWaw=cw@mail.gmail.com>
 <CAD21AoCdx5ZNS_cO7bYz1Zfb+Kw1kuJV2wtewrz7T1pPpjcWGw@mail.gmail.com>
 <CAJDiXgi6ZQOoSEqj9RyZMEh+HHBtmW0+PHD85UNPtKch8ubvdg@mail.gmail.com>
 <CAD21AoBcoA-i-pJ_=y+jg14R8_QaJA1iwktCnu5i-C=yXDFPdA@mail.gmail.com>
 <CAJDiXgjnUdE6Sk4M0unmT+9dULyFAxcum2txQKpWTuo4uQ_oXQ@mail.gmail.com>
 <CAD21AoBTZdVR93JBo620B=MX-K8cdm3VRbjrBr_Vcpngk3AjVw@mail.gmail.com>
 <CAA5RZ0vfBg=c_0Sa1Tpxv8tueeBk8C5qTf9TrxKBbXUqPc99Ag@mail.gmail.com>
 <CAD21AoBgvUeWS8ZsXBahA1XdYayK6DJ6dx49d6Xpii-iH+Hrwg@mail.gmail.com>
 <CAA5RZ0vF+Lr-jU1LAZWTGUjboUETk8oLvaNBbA5ozX6dau+how@mail.gmail.com>
 <CAJDiXggueLSGMNRmLshbmFRfbo4jzks0W8bLDfUSRZ-61fPVEQ@mail.gmail.com>
 <CAFY6G8cJ=DRTX75pOGerH6sk39dRt+7MSH+y_qppDdhPs=qdQA@mail.gmail.com>
 <CAJDiXgg1t6wk9NjyMUTm1iKqM9GtdQ_wrEchBtz3xjWBZM8W8A@mail.gmail.com>
 <CAD21AoAC0=Xi38RQcAO4A+vdmoXToZMoHfbS=KLT49fAOTH_gA@mail.gmail.com>
 <CAJDiXgiD+AZKhJSn-FSRVQxtDLmJd95wDu4wtKniQF5==1JcjQ@mail.gmail.com>
 <CAD21AoAM8KsqNhrZYJuf7odvxcTC0TumXazJc-r_wC5KnDFDPg@mail.gmail.com>
 <CAJDiXghbcOC9OOj3ampxuyqXH0geggnosnrYUHGygkpss-RtxA@mail.gmail.com>
 <CAD21AoAPnq0vrcGgeN++r1GoL8Kza7jaGL=TNzuBn6+MkR=rUQ@mail.gmail.com>
 <CAJDiXghmsbTmnm--9B5bbuZXa1OL7SZ0HYppX3tx9XsdwfJBhA@mail.gmail.com>
 <DB3C67FCRLOO.1R5NLYCNEA6BF@gmail.com>
 <CAJDiXgiYiX+azuR76DcVx8fZn57m_4v6cB14-GW34mWa=qudFQ@mail.gmail.com>
 <CAD21AoDtPpkkQ_h1yf4oTx1qn4SRdTeVY3qs+9J07fYqa_4Gww@mail.gmail.com>
 <CAJDiXgi7KB7wSQ=Ux=ngdaCvJnJ5x-ehvTyiuZez+5uKHtV6iQ@mail.gmail.com>
 <CAD21AoCcHKKXsr9Oh736ejckqqS1i430xGEyJ=JP5OL0ExyP1A@mail.gmail.com>
 <CAJDiXghaFT_1sSv3q8mjyZ_RLZDgiogg0mWRvLxSWvkUi2CcLg@mail.gmail.com>
 <CAA5RZ0u63W41OmcEO+HLs4CSo-Sd3J+Q-4=04iud8V=xX4iUrA@mail.gmail.com>
 <CAJDiXgin1TXniVGJKzOTA=F9K342uVfm6O0EmubTVB=F+XSrbA@mail.gmail.com>
 <CAD21AoDadzAwibxf-+urjx=XL+eVu8=Ut-Lh2GxXUt32LbPG3Q@mail.gmail.com>
 <CAD21AoD6HhraqhOgkQJOrr0ixZkAZuqJRpzGv-B+_-ad6d5aPw@mail.gmail.com>
 <CAJDiXgiGSpqMQSOx-cVO_LtcB5GWHBy9ph7oOR4ebbX8A==kgw@mail.gmail.com>
 <CAD21AoBRRXbNJEvCjS-0XZgCEeRBzQPKmrSDjJ3wZ8TN28vaCQ@mail.gmail.com>
 <CAPpHfduBJfMcojvmYHUo8b_C=0cxRy1N+tNiNGoA3RAZq2ApaA@mail.gmail.com>
 <CAD21AoC82NeHKXc965pPUZO2eyo1U7P6cmfRJbrcPDcnd7_6hw@mail.gmail.com>
 <CAJDiXghP2kXnEz+cj3rAWNM3NdKSB_4WtnngFXpVz2omPhGr5A@mail.gmail.com>
 <CAD21AoA0bnRZC_OqKMnH-Ln+OZ9z9k56j2c_MXj8pw69O-wkBw@mail.gmail.com>
 <CAA5RZ0sSXDza7_nUUbhHL_Sws+M+HR1daKJPXHpdLuNCkwUgUg@mail.gmail.com>
 <CAJDiXggrBsbzOisf+Nu8pZkYGrpUZaFbosL1Wbm3kKxzTm4xgw@mail.gmail.com>
 <CAA5RZ0tbiPcgQEjnhdnjz6qSjfRsGrr8jGCaMcrMaoPpax3wig@mail.gmail.com>
 <CAJDiXgjt5ZmK2uvS0E8Ztt5ePYmq8Ze_dG05Zo2NUsKLHCEuYA@mail.gmail.com>
 <CAD21AoB7v5tLPXLK=qmtt6PaEC1f+Fb-gh+MwAbXfm6x4eZGNw@mail.gmail.com>
 <CAJDiXghwtUbiFnAh3nSaxTk8KFupQuMbp+g4z3wOLoQfMuqgDg@mail.gmail.com>
 <CAJDiXgjoNd4BF19HNY_FAcDUqiqsfw8cGhNOJwBxahB8P38E3Q@mail.gmail.com>
 <CAD21AoBT1LWqPZkcHpVMVh0ZOXUneO=p61t0i8cQ+kOP9qfODQ@mail.gmail.com>
 <CAJDiXggL=J0nV7PfBsMW9+UOU3KUp1jNBM9Gov1JvAX7aG_U1g@mail.gmail.com>
 <CAD21AoDz-1Zf9DOJJrdcB2=eNA4UdywthkowNp_dHmOGC-yV_g@mail.gmail.com>
 <CAJDiXgjzphJ313=aDwbvryHpmTi6AqE+-5crysTtzKv01-vkzA@mail.gmail.com>
 <CAD21AoD7_4gsQ2a82zO3SaRwjdw_3tyiYDHNFPUKQ5DAA5HOtA@mail.gmail.com>
 <CAJDiXggY1QzNde6_HhpzneLc9dYqmWZ+PY39cuBXYdcCTuoJBA@mail.gmail.com>
 <CAD21AoCFPiS2jcMA1JaV1kT8xrGz5BpN7iBP_gCgRuaANEbciA@mail.gmail.com>
 <CAJDiXgh6jmNGR3uOB_6YeGhNkR2=HdTdEYjmHXdumNzyY4MckQ@mail.gmail.com>
 <CAD21AoDs3SOXeAEoCRizfEKybpRkE7t7poX0+iZ6MM1MFWMsfA@mail.gmail.com>
 <CAJDiXgjTkuqSPerC_nasxDz6d2Komf1ipYKV6SupDRnc9yhO9w@mail.gmail.com>
 <CAD21AoAXMjX03h5K84u0heBLU+fqGgWBGBDwnBDGSs=DhyF9pQ@mail.gmail.com>
 <CAJDiXghjZEAYboGhujgGvY9=RiFD01ERHVVF+NQMuuAKVZDmDQ@mail.gmail.com>
 <CAD21AoAD+N5SxBr0qL7TeWnvq4iYmFT=DyWdNLQPB-XntYkwEg@mail.gmail.com>
 <CAJDiXgjgn87sH1-MmONPKkeYJG83C0ChrYkYn9UcRonLhOOfOw@mail.gmail.com>
In-Reply-To: 
 <CAJDiXgjgn87sH1-MmONPKkeYJG83C0ChrYkYn9UcRonLhOOfOw@mail.gmail.com>
From: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Wed, 11 Mar 2026 12:05:16 -0700
Message-ID: 
 <CAD21AoCoJYauWO78M-CGdHpYfcqEZVV5a1Z-7wWB=-G-x8EVFg@mail.gmail.com>
Subject: Re: POC: Parallel processing of indexes in autovacuum
To: Daniil Davydov <3danissimo@gmail.com>
Cc: Sami Imseih <samimseih@gmail.com>,
 Alexander Korotkov <aekorotkov@gmail.com>,
	Matheus Alcantara <matheusssilv97@gmail.com>,
 Maxim Orlov <orlovmg@gmail.com>,
	Postgres hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/CAD21AoCoJYauWO78M-CGdHpYfcqEZVV5a1Z-7wWB%3D-G-x8EVFg%40mail.gmail.com>
Precedence: bulk

On Wed, Mar 11, 2026 at 4:28=E2=80=AFAM Daniil Davydov <3danissimo@gmail.co=
m> wrote:
>
>
> > While I agree that showing only two numbers might lack some
> > information for users, I guess the same is true for
> > max_parallel_maintenance_workers or other parallel queries related to
> > GUC parameters. For instance, suppose we set
> > max_parallel_maintenance_workers to 2, if the table has (large enough)
> > 4 indexes, we would plan to execute a parallel vacuum with 2 workers
> > instead of 4 due to max_parallel_maintenance_worker shortage and it's
> > even possible that only 1 worker can launch due to
> > max_worker_processes shortage. In this case, we currently consider
> > that 2 workers are planned. Isn't it the same situation as the case
> > where we reserved 2 parallel vacuum workers for autovacuum for the
> > table with 4 indexes?
>
> I don't think that examples with other "max_parallel_" parameters will be
> appropriate, because these parameters are limiting the number of parallel
> workers for *single* operation/executor node/... .  At the same time,
> av_max_parallel_workers limits the total number of parallel workers acros=
s
> all a/v leaders.
>
> Regarding the situation that you provided :
> The number of planned workers is reduced inside the
> parallel_vacuum_compute_workers due to the max_parallel_maintenance_worke=
rs
> limit.  I.e. we cannot plan more workers than required by the config, and
> it's completely OK No one expects the number of "planned workers" to be m=
ore
> than max_parallel_maintenance_workers.
>
> IMO there is no need to make efforts to track the shortage of
> max_parallel_maintenance_workers for the VACUUM (PARALLEL), because this
> parameter just plays the role of a limiter. We will consider only the
> shortage of max_parallel_workers, that can be determined by looking at
> "planned vs. launched".
>
> And here is a difference with a parallel autovacuum :
> av_max_parallel_workers is considered twice : in the
> "parallel_vacuum_compute_workers" and "ReserveWorkers" functions.
> So the low number of launched workers can be explained by the shortage of
> both av_max_parallel_workers and max_parallel_workers. Since we want to
> distinguish between these cases, we have added the "nreserved" concept.
>
> I see that few modules can report something like "out of background worke=
r
> slots" when they cannot launch more workers due to max_parallel_workers
> shortage (but modules depending on the "parallel.c" logic don't do so).
> This fact gave me another idea :
> If we don't want to log "nreserved" or some other similar value, maybe
> we should add logging after the "ReserveWorkers" function? I.e. if some
> workers cannot be reserved, we can emit a log like "out of parallel
> autovacuum workers. you should increase the av_max_parallel_workers
> parameter". Having this log can help the user distinguish between
> max_parallel_workers/av_max_parallel_workers shortage situations.
> What do you think?

My point is that the process of determining the number of workers
planned to launch is somewhat unclear to users in both cases. We
consider not only GUCs such as max_parallel_maintenance_workers but
also index AM definitions (i.e., amparallelvacuumoption) and index
sizes etc. But I agree that providing more detailed logs might help
users understand and notice the av_max_parallel_workers shortage.

BTW thes discussion made me think to change av_max_parallel_workers to
control the number of workers per-autovacuum worker instead (with
renaming it to say max_parallel_workers_per_autovacuum_worker). Users
can compute the maximum number of parallel workers the system requires
by (autovacuum_worker_slots *
max_parallel_workers_per_autovacuum_worker). We would no longer need
the reservation and release logic. I'd like to hear your opinion.

>
> Summary :
> 1)
> I think that we should not look at maintenance vacuum while
> considering how to inform the user about parameters shortage for autovacu=
um,
> because we have a more complicated situation in case of autovacuum.
> 2)
> I suggest adding a separate log that will be emitted every time we are
> unable to start workers due to a shortage of av_max_parallel_workers.

For (2), do you mean that the worker writes these logs regardless of
log_autovacuum_min_duration setting? I'm concerned that the server
logs would be flooded with these logs especially when multiple
autovacuum workers are working very actively and the system is facing
a shortage of av_max_parallel_workers.

>
> > * 0004 patch:
> >
> > Can we write the same test cases while not relying on the 0002 patch
> > (i.e., worker usage logging)? We check the worker usage log at two
> > places in the regression tests. The idea is that we write the number
> > of workers planned, reserved, and launched in DEBUG log level and
> > check these logs in the regression tests. The patch 0001, 0003, and
> > 0004 can be merged before push while we might want more discussion on
> > the 0002 patch.
>
> Possibly we can introduce a new injection point, or a new log for it.
> But I assume that the subject of discussion in patch 0002 is the
> "nreserved" logic, and "nlaunched/nplanned" logic does not raise any
> questions.
>
> I suggest splitting the 0002 patch into two parts : 1) basic logic and
> 2) additional logic with nreserved or something else. The second part can=
 be
> discussed in isolation from the patch set. If we do this, we may not have=
 to
> change the tests. What do you think?

Assuming the basic logic means nlaunched/nplanned logic, yes, it would
be a nice idea. I think user-facing logging stuff can be developed as
an improvement independent from the main parallel autovacuum patch.
It's ideal if we can implement the main patch (with tests) without
relying on the user-facing logging.

Regards,

--=20
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com