Feedback-ID: i0fe9450f:Fastmail
Date: Sun, 18 Dec 2022 11:21:47 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: postgresql@taljaren.se, pgsql-bugs@lists.postgresql.org,
	Peter Geoghegan <pg@bowt.ie>
Subject: Re: BUG #17717: Regression in vacuumdb (15 is slower than 10/11 and
 possible memory issue)
Message-ID: <Y555O1zrtHOshuRC@paquier.xyz>
References: <17717-6c50eb1c7d23a886@postgresql.org>
 <3439037.1671130590@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="bY3Hbk/q2oUqIsJA"
Content-Disposition: inline
In-Reply-To: <3439037.1671130590@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/Y555O1zrtHOshuRC%40paquier.xyz>
Precedence: bulk


--bY3Hbk/q2oUqIsJA
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Thu, Dec 15, 2022 at 01:56:30PM -0500, Tom Lane wrote:
> * To fix vacuumdb properly, it might be enough to get it to
> batch VACUUMs, say by naming up to 1000 tables per command
> instead of just one.  I'm not sure how that would interact
> with its parallelization logic, though.  It's not really
> solving the O(N^2) issue either, just pushing it further out.

I have been thinking about this part, and using a hardcoded rule for
the batches would be tricky.  The list of relations returned by the
scan of pg_class are ordered by relpages, so depending on the
distribution of the sizes (few tables with a large size and a lot of
table with small sizes, exponential distribution of table sizes), we
may finish with more downsides than upsides in some cases, even if we
use a linear rule based on the number of relations, or even if we
distribute the relations across the slots in a round robin fashion for
example.

In order to control all that, rather than a hardcoded rule, could it
be as simple as introducing an option like vacuumdb --batch=N
defaulting to 1 to let users control the number of relations grouped
in a single command with a round robin distribution for each slot?
--
Michael

--bY3Hbk/q2oUqIsJA
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmOeeToACgkQnvQgOdby
QH1/1A//f/tSiADj3mhbzkKT9bHmKfvN80EC7eRqstK3PhWXeRlbgkBiGZBwlPta
UtROCfOnw0sCaJZ1nhV0oq/XYQ3bLC+QreHsEXBQCxUM6PeX/KgbED0H51dds0Md
hRfgxMqtR0AlT7KAOrt+MUKLM1SV7nEEzcd8yy409mfkGC5aJxZAQqb86ICsAOUT
9k+dLtPeVFeIwBC8yxYqJ7/uT5I1FITaQIaT+t0TOIB4+IHVJUTCfK9pZw/ylfAi
bs6xaPQZLAg0FwFoftvF6QoQiIW5RP+01GK8df8P03vdPK6A4OXaBP3TN17gMRbV
8jKZieSeBL2uqBZ/OIFwdvV9pb2sJapZi1Tjwt23Nxw38+Vrn/S3OsRPgcEsfOJR
wUAR2nBRhW6NP151ZfoB55Cotm4+W6HOYcztFe1Us27/cEtpl4Vfnx2JuJqp1J+j
2mC6biAlH1r21EWAvMFR05/Gf5y/W9zwy2uA0hytTFXCWA4cGAOoRyLX5pYJM8tz
8PD1fKbkYylxHBx1H8euHHnyNKWcJYWWBUJE/bkGQuFNYc1xGACQso7TZUSFwUgr
r0+IJMVV8YTyY9ZwwLh5OqqE+0EMFwOOP/2MI1GCEzSsBBYUZEeMCIcra/4/ssOV
OoTcUpxcVjBbJz0Okn4Qx4EMxJJLUEM8knbBgMgc+npjhEE17W4=
=hcHb
-----END PGP SIGNATURE-----

--bY3Hbk/q2oUqIsJA--