Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1w7To6-005MEB-1z for pgsql-hackers@arkaria.postgresql.org; Tue, 31 Mar 2026 07:46:38 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1w7To5-008c1D-0I for pgsql-hackers@arkaria.postgresql.org; Tue, 31 Mar 2026 07:46:37 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1w7To4-008c14-2N for pgsql-hackers@lists.postgresql.org; Tue, 31 Mar 2026 07:46:37 +0000 Received: from mail-vk1-xa33.google.com ([2607:f8b0:4864:20::a33]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1w7To2-000000028MJ-2Mtp for pgsql-hackers@lists.postgresql.org; Tue, 31 Mar 2026 07:46:36 +0000 Received: by mail-vk1-xa33.google.com with SMTP id 71dfb90a1353d-56b7043c97eso2082819e0c.1 for ; Tue, 31 Mar 2026 00:46:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1774943193; cv=none; d=google.com; s=arc-20240605; b=hoZzNZNMmPoCPbzGTi5FodnN/mq3JM4HOPPV/2+/rfIV4FvvB0CokZPg+bGYloUnCA jxseOCy4ycq2Mfm5Aa8fY8MKR+vNTJg38DOFQtAkOQbc9vgMo+B90QLRMHcrwXXHk7lD hY74viT+jVmyL7gF/UbH5BuVSdDG2Nw/Jen+Y34LtmUqOwwoUdassrupeoPS+lriYO+A goC9WYcO02vlQ7WlQmu4+byCp0tZjIyl2+V7GnNyPOFD/lef7qGmSz6X+3bsa7WtZnKO MF2Vy3TpUG+TBtOrS+Rkzl3AecbVhYIvsgy9UdVXC1HSWaL5jnHMJcFEAedZMf0HwIJp 45eA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=CjWPnLf92AmTwXe8iTsbt4Y4RT9C9s9vB5JI1RCXYqc=; fh=upYwAFD954m6tNr2gHvD0WQRBTH4irVjOjhpLOVwg5c=; b=arM4gl+CUlnYsyyR9UUIvxAZcXgXwu6bJvXFqlDhj1lB/2aNLN71+VEU5Tbm+af1Zv hOloDdY2u0a6SeClpVj3BZmVYfuJm6gkKDLRrqwJdmMeHzn1Zww667TH6VPr0jmcMCrN lxqX5BFtMK9RXOJjevAUf9FUZVd9eCtRqA/IVtbEL1F9DXgHulrMbhhu1dYt9y+WgDt+ MbfV6Am4LjV7K50WHA7/6zABPm+gBeOhYb0XaPYywL7FMq+OrSp80qlC8wAjT8jAOH+2 abW2BYTwrzpcUOr97/jmUR1YObSgTecmMVvLw4g0QXYyVN0/s7FkEN0SHtEvlsOnEX7W wGnw==; darn=lists.postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774943193; x=1775547993; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=CjWPnLf92AmTwXe8iTsbt4Y4RT9C9s9vB5JI1RCXYqc=; b=mW9sPptFo+aHpY/4cywRu212ph2K9wZUfRFcwjSpoOotDRQvUCLngqw8oszD2uMdcB 56qC3cE+7Vc+mDzFiWSMt+KxEoQqdhV5oQSxHub+CDvnBvpaBpXiArhSOM/WERDVEmgr XWgBIMTEsXPfC9jUc9p2n8gzRMIXQnn+MCkfz8IeyRX0LnvX+NLmgHwIM5jr+Qufckym kSotSmQdGePxOHPAXE/TveIXkPxAZL/O88JTlCwPwFLwWuuz8wB7Hpz8eoln3vzoTESv sM7hPUEFvlGUaPcZnFtzlvnCp8Gxlq39GGXR0ZTidgB2sLs+FklnKiF++ulhPshhDImN /gfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774943193; x=1775547993; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=CjWPnLf92AmTwXe8iTsbt4Y4RT9C9s9vB5JI1RCXYqc=; b=sT1P33YAJXgp7MjHgOs+a05/wBv9HqvzY8xBFHj3mDpZO2NpaDE7z2vC+HeN91ahFL iPsDS7lhCOWlU/iIwULweoilaAYei0jjpgpXVZV8jkaSjSXAc/FiMpuekkuWv1PneoZK 3RZhCV7BUh1XmTuSq7LUqNQ6TQXkRYrUCllHsNKF2JOYXcHrmvDR6Y8onpUZYTtB4sNK 5sP01yL1jj+rYcqhXaj6m8OZSF2rLE0IEGpX38oAwT3QIBmxz4D582REh0uE5mqjvLMe SG2VacqlRQRbzA6dz13AhDA2M8YmPOdAFQvFWZ9htXH7JenQaNGWcNLSOQSQKwuu3SfU G7YQ== X-Forwarded-Encrypted: i=1; AJvYcCW7JkMVJSYAmUL7PtF9mqrqyFNCMlBkeqJqas5gHxTDvtMZQJpaZ1N6Pa0ZKQOFSBhdGlWmv+bLRzWWmdQG@lists.postgresql.org X-Gm-Message-State: AOJu0YwefO+Win03nBPB7oTZqUntx9w4wGbVTsvMaaSxw5sCaZ7lnJ5Y bV/mUm+lOSJVoKwe13zvL02rtdtnG66ejGUCF2Wh5si63NHzN8G5DPpcanpg8xcGJra4joMdM4q swB8oGKZGcJShewKNL5nq0UUNAoE2RC4= X-Gm-Gg: ATEYQzwHvgESZDExaHMTz1fceTu97FKzZCg5rAapKEdmSGoJSoUzh3bYPLjWaK1wwxw F1h2WPbRBtiVx4X8GpjeyRDJaemlxdibXSX/MGVuFsVKpOBuX5RNWKrnjBowl8Loz5Wy24aqkbc YiVQpMZcfTyjEY5Fnz3kDj9tyCDNHhI79NltjudJiH+Bf4zdAO1OAJ97bUoBm43hgsPBbAcKHyN JZDGBosrMbOgd28VDWw5WVEQL0qFaCAmGkcdgjNqkLBcNhaYgMhRq82EP4dnwjAv66NZbxX2Oj7 GsXcqBA= X-Received: by 2002:a05:6122:659e:b0:56b:8d2a:8c8f with SMTP id 71dfb90a1353d-56d4a606581mr6523033e0c.11.1774943193105; Tue, 31 Mar 2026 00:46:33 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: SATYANARAYANA NARLAPURAM Date: Tue, 31 Mar 2026 00:46:20 -0700 X-Gm-Features: AQROBzBFevq0wjI4O4-hGRmTPF-dj6zZZr38edt_VZfUBjeIyRr2VL9lEF_kCrw Message-ID: Subject: Re: POC: Parallel processing of indexes in autovacuum To: Daniil Davydov <3danissimo@gmail.com> Cc: Bharath Rupireddy , Masahiko Sawada , Sami Imseih , Alexander Korotkov , Matheus Alcantara , Maxim Orlov , Postgres hackers Content-Type: multipart/alternative; boundary="00000000000085886d064e4d2a32" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --00000000000085886d064e4d2a32 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi On Mon, Mar 30, 2026 at 1:44=E2=80=AFAM Daniil Davydov <3danissimo@gmail.co= m> wrote: > Hi, > > On Mon, Mar 30, 2026 at 7:17=E2=80=AFAM SATYANARAYANA NARLAPURAM > wrote: > > > > Thank you for working on this, very useful feature. Sharing a few > thoughts: > > > > 1. Shouldn't we also cap by max_parallel_workers to avoid wasting DSM > resources in parallel_vacuum_compute_workers? > > Actually, autovacuum_max_parallel_workers is already limited by > max_parallel_workers. It is not clear for me why we allow setting this GU= C > higher than max_parallel_workers, but if this happens, I think it is a > user's > misconfiguration. Isn=E2=80=99t there a wasted effort here if user misconfigures because anyw= ay we cannot launch that many workers? I suggest making a check here. > > > > 2. Is it intentional that other autovacuum workers not yield cost limit= s > to the parallel auto vacuum workers? Cost limits are distributed first > equally to the autovacuum workers. > > and then they share that. Therefore, parallel workers will be heavily > throttled. IIUC, this problem doesn't exist with manual vacuum. > > If we don't fix this, at least we should document this. > > Parallel a/v workers inherit cost based parameters (including the > vacuum_cost_limit) from the leader worker. Do you mean that this can be t= oo > low value for parallel operation? If so, user can manually increase the > vacuum_cost_limit reloption for those tables, where parallel a/v sleeps t= oo > much (due to cost delay). They don=E2=80=99t inherit but share, isn=E2=80=99t it? > > BTW, describing the cost limit propagation to the parallel a/v workers is > worth mentioning in the documentation. I'll add it in the next patch > version. Yes, that helps > > > > 3. Additionally, is there a point where, based on the cost limits, > launching additional workers becomes counterproductive compared to runnin= g > fewer workers and preventing it? > > I don't think that we can possibly find a universal limit that will be > appropriate for all possible configurations. By now we are using a pretty > simple formula for parallel degree calculation. Since user have several > ways > to affect this formula, I guess that there will be no problems with it > (except > my concerns about opt-out style). Thanks, Satya > > --00000000000085886d064e4d2a32 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi

=
On Mon, Mar 30, 2026 at 1:44=E2=80=AF= AM Daniil Davydov <3danissimo@gm= ail.com> wrote:
Hi,

On Mon, Mar 30, 2026 at 7:17=E2=80=AFAM SATYANARAYANA NARLAPURAM
<satyanar= lapuram@gmail.com> wrote:
>
> Thank you for working on this, very useful feature. Sharing a few thou= ghts:
>
> 1. Shouldn't we also cap by max_parallel_workers to avoid wasting = DSM resources in parallel_vacuum_compute_workers?

Actually, autovacuum_max_parallel_workers is already limited by
max_parallel_workers. It is not clear for me why we allow setting this GUC<= br> higher than max_parallel_workers, but if this happens, I think it is a user= 's
misconfiguration.

Isn=E2=80=99t there a wasted effort here if user misconfigures because any= way we cannot launch that many workers? I suggest making a check here.


> 2. Is it intentional that other autovacuum workers not yield cost limi= ts to the parallel auto vacuum workers? Cost limits are distributed first e= qually to the autovacuum workers.
> and then they share that. Therefore, parallel workers will be heavily = throttled. IIUC, this problem doesn't exist with manual vacuum.
>=C2=A0 If we don't fix this, at least we should document this.

Parallel a/v workers inherit cost based parameters (including the
vacuum_cost_limit) from the leader worker. Do you mean that this can be too=
low value for parallel operation? If so, user can manually increase the
vacuum_cost_limit reloption for those tables, where parallel a/v sleeps too=
much (due to cost delay).

They don=E2=80=99t inherit but share, isn=E2=80=99t it?



BTW, describing the cost limit propagation to the parallel a/v workers is worth mentioning in the documentation. I'll add it in the next patch ve= rsion.

Yes, that = helps


> 3. Additionally, is there a point where, based on the cost limits, lau= nching additional workers becomes counterproductive compared to running few= er workers and preventing it?

I don't think that we can possibly find a universal limit that will be<= br> appropriate for all possible configurations. By now we are using a pretty simple formula for parallel degree calculation. Since user have several way= s
to affect this formula, I guess that there will be no problems with it (exc= ept
my concerns about opt-out style).

<= div dir=3D"auto">Thanks,
Satya

--00000000000085886d064e4d2a32--