Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tXWMl-008noL-F6 for pgsql-hackers@arkaria.postgresql.org; Tue, 14 Jan 2025 02:09:16 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1tXWMj-00Eett-8P for pgsql-hackers@arkaria.postgresql.org; Tue, 14 Jan 2025 02:09:13 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tXWMi-00Eetl-S1 for pgsql-hackers@lists.postgresql.org; Tue, 14 Jan 2025 02:09:13 +0000 Received: from mail-io1-xd29.google.com ([2607:f8b0:4864:20::d29]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1tXWMg-000H5W-2W for pgsql-hackers@lists.postgresql.org; Tue, 14 Jan 2025 02:09:12 +0000 Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-844e9b8b0b9so386483939f.0 for ; Mon, 13 Jan 2025 18:09:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736820549; x=1737425349; darn=lists.postgresql.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=w3TWTPrqmTxWDFwjCK0pRc9y0jl5SuPcewjP6w3KL7k=; b=RdZXHOYkJ2ncB9QEk5Zjgee+QakGaSeNJkkPPb7A7f6hSGVnh2L4nZehPvOOkOLChj ogwyVVVnHgUIV3ynN5NK0LWD1nXCS0MJ5yaFcopAUOsEFKGlfbTZExGxk3AjIM2UghDl OpZ8stSPkJcLp68H5HPS30Y/uVRIt4KmmaaV5R4tq3Z0UL287WyJz3mqtESfPpiLReko wPTeE5NYny0siepkLk0cab+0YevY7C2smtj0Z4Uq4F0fIpU6gIYmQNmLy0LPy3zMSGDm Cx+OY/KlBYH7ObsYJ/gnaotDRJDh1wJx8CvYwXARZGiru6aFlmBxXUtwj8ZBrpesJxf+ xxSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736820549; x=1737425349; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=w3TWTPrqmTxWDFwjCK0pRc9y0jl5SuPcewjP6w3KL7k=; b=hnlI9OrOu0tSFfkCCeqqjCV1547zxV2melniZvHnqAYSErQhkeJ66pZSsXG3h4fLmg MFP+4lmpXn06jApfVhWFa5p+k01Re+g7V4rJtKq8HM/pp0G08XYnJnNcCRvcRFGASqNa w0gJYdHOzKXhOorzIp3n0TadgQTRlolhb8Yuii+f/7qAcTJenpfSd40WWib2yWAjIY+j kDpzIGo2HwqrJUZF3j1pTm6R9V++mMVjgAK5IcIFaRPBLP7ssxJKEILJemqgUfFCesM/ d8jBbJ10hODw7T+Fnz1QaQQKw2k/hBtdy4h2qLeA6ahUYZ7QyuMXUka5yMx5c1QKEf/C xLNg== X-Forwarded-Encrypted: i=1; AJvYcCUw6rs3jdTnIhYpw1Y8l8Gqc7fF8KVvR0FyBd8MnQb5yyrTsuJhqFFZTePID9QV3EmvLVhVvlrJ35pr0E8C@lists.postgresql.org X-Gm-Message-State: AOJu0YxWl26jRTXSwE7DEBqkINAcFYnQTiEpnZ9JF5bnvUqyagvjGgGU kPt/Gzq9CualWVUpJY0nnBGvGAtxNLcltlhRJDmSv9xKsqKZDyKc X-Gm-Gg: ASbGncsVI12n2KJpIzMDxV9NeL4Fxal8SMck7emZSBMlz7J8YgdHKjxLUOa8sYTLyJl 8RNn5PlYoNAuE8E+tXVf4dcsv3py5vWHSc/R17VfZROEG6lBM1lY5YrMjWY5x6XPLLowBdXk4h2 OkJ72taiZddX2sCRchnFeB4/Ays00qfO886a8HldSmCzKEhDRxCW3DazEbdRDen70aTQcPB3mwi n0RAXfcRssMxU8Xs7eqyNIRIbhgFDSodv6CYFlACCKqeJpVpHCqSdcEsWbBADZ9SCHezLXj1hD6 vC15mg3KO94XapZXLUOn0iDQ1Lurma4Z6LZ6Eytk6Fl5hlk= X-Google-Smtp-Source: AGHT+IGZ9khEMX5F6pr5NycNhUnogLkBQ/hZ5gMtGuzieadYoLK/PCjQY1jl/RcQMR1CYl3I0bfzAQ== X-Received: by 2002:a05:6602:6402:b0:84f:5547:8398 with SMTP id ca18e2360f4ac-84f5547866emr208266739f.11.1736820548900; Mon, 13 Jan 2025 18:09:08 -0800 (PST) Received: from nathan (162-195-168-172.lightspeed.stlsmo.sbcglobal.net. [162.195.168.172]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-84d61fa3ce0sm304792239f.38.2025.01.13.18.09.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jan 2025 18:09:07 -0800 (PST) Date: Mon, 13 Jan 2025 20:09:05 -0600 From: Nathan Bossart To: Sami Imseih Cc: Robert Treat , =?iso-8859-1?Q?Fr=E9d=E9ric?= Yhuel , wenhui qiu , Robert Haas , "Imseih (AWS), Sami" , David Rowley , Joe Conway , Michael Banck , Laurenz Albe , Melanie Plageman , PostgreSQL Hackers Subject: Re: New GUC autovacuum_max_threshold ? Message-ID: References: <2ca7f9c7-5da9-46cb-9072-39b71bfab948@dalibo.com> <93f3dab5-caa4-4213-9be0-855d5aaa19bf@dalibo.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="YwvA4pfL6umvmJzT" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --YwvA4pfL6umvmJzT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Jan 13, 2025 at 05:17:11PM -0600, Sami Imseih wrote: > I propose renaming the GUC from "autovacuum_max_threshold" to > "autovacuum_vacuum_max_threshold" to clarify that it applies only > to the vacuum operation performed by autovacuum, not to the analyze operation. > This will also align with naming for other related GUCs, i.e., > "autovacuum_analyze_threshold" and "autovacuum_vacuum_threshold." > > The "vacuum threshold" calculation described in [1] will also need to be > updated. Good call. Here is an updated patch. -- nathan --YwvA4pfL6umvmJzT Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: attachment; filename="v6-0001-Introduce-autovacuum_vacuum_max_threshold.patch" Content-Transfer-Encoding: 8bit From 1a06dd9ccc2ee67a60278561f16c2c76cf3f8dc5 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 13 Jan 2025 20:01:24 -0600 Subject: [PATCH v6 1/1] Introduce autovacuum_vacuum_max_threshold. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Author: Nathan Bossart, Frédéric Yhuel Reviewed-by: Melanie Plageman, Robert Haas, Laurenz Albe, Michael Banck, Joe Conway, Sami Imseih, David Rowley, wenhui qiu Discussion: https://postgr.es/m/956435f8-3b2f-47a6-8756-8c54ded61802%40dalibo.com --- doc/src/sgml/config.sgml | 24 +++++++++++++++++++ doc/src/sgml/maintenance.sgml | 6 +++-- doc/src/sgml/ref/create_table.sgml | 15 ++++++++++++ src/backend/access/common/reloptions.c | 11 +++++++++ src/backend/postmaster/autovacuum.c | 12 ++++++++++ src/backend/utils/misc/guc_tables.c | 9 +++++++ src/backend/utils/misc/postgresql.conf.sample | 3 +++ src/bin/psql/tab-complete.in.c | 2 ++ src/include/postmaster/autovacuum.h | 1 + src/include/utils/rel.h | 1 + 10 files changed, 82 insertions(+), 2 deletions(-) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 3f41a17b1fe..54a1ec2084a 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -8579,6 +8579,30 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; + + autovacuum_vacuum_max_threshold (integer) + + autovacuum_vacuum_max_threshold + configuration parameter + + + + + Specifies the maximum number of updated or deleted tuples needed to + trigger a VACUUM in any one table, i.e., a cap on + the value calculated with + autovacuum_vacuum_threshold and + autovacuum_vacuum_scale_factor. The default is + 100,000,000 tuples. If -1 is specified, autovacuum will not enforce a + maximum number of updated or deleted tuples that will trigger a + VACUUM operation. This parameter can only be set + in the postgresql.conf file or on the server + command line; but the setting can be overridden for individual tables + by changing storage parameters. + + + + autovacuum_vacuum_insert_threshold (integer) diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml index 0be90bdc7ef..f84ad7557d9 100644 --- a/doc/src/sgml/maintenance.sgml +++ b/doc/src/sgml/maintenance.sgml @@ -895,9 +895,11 @@ HINT: Execute a database-wide VACUUM in that database. VACUUM exceeds the vacuum threshold, the table is vacuumed. The vacuum threshold is defined as: -vacuum threshold = vacuum base threshold + vacuum scale factor * number of tuples +vacuum threshold = Minimum(vacuum max threshold, vacuum base threshold + vacuum scale factor * number of tuples) - where the vacuum base threshold is + where the vacuum max threshold is + , + the vacuum base threshold is , the vacuum scale factor is , diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index 2237321cb4f..417498f71db 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -1712,6 +1712,21 @@ WITH ( MODULUS numeric_literal, REM + + autovacuum_vacuum_max_threshold, toast.autovacuum_vacuum_max_threshold (integer) + + autovacuum_vacuum_max_threshold + storage parameter + + + + + Per-table value for + parameter. + + + + autovacuum_vacuum_scale_factor, toast.autovacuum_vacuum_scale_factor (floating point) diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c index e587abd9990..5731cf42f54 100644 --- a/src/backend/access/common/reloptions.c +++ b/src/backend/access/common/reloptions.c @@ -231,6 +231,15 @@ static relopt_int intRelOpts[] = }, -1, 0, INT_MAX }, + { + { + "autovacuum_vacuum_max_threshold", + "Maximum number of tuple updates or deletes prior to vacuum", + RELOPT_KIND_HEAP | RELOPT_KIND_TOAST, + ShareUpdateExclusiveLock + }, + -2, -1, INT_MAX + }, { { "autovacuum_vacuum_insert_threshold", @@ -1843,6 +1852,8 @@ default_reloptions(Datum reloptions, bool validate, relopt_kind kind) offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, enabled)}, {"autovacuum_vacuum_threshold", RELOPT_TYPE_INT, offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, vacuum_threshold)}, + {"autovacuum_vacuum_max_threshold", RELOPT_TYPE_INT, + offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, vacuum_max_threshold)}, {"autovacuum_vacuum_insert_threshold", RELOPT_TYPE_INT, offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, vacuum_ins_threshold)}, {"autovacuum_analyze_threshold", RELOPT_TYPE_INT, diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c index 0ab921a169b..4b1e42635d0 100644 --- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -120,6 +120,7 @@ int autovacuum_max_workers; int autovacuum_work_mem = -1; int autovacuum_naptime; int autovacuum_vac_thresh; +int autovacuum_vac_max_thresh; double autovacuum_vac_scale; int autovacuum_vac_ins_thresh; double autovacuum_vac_ins_scale; @@ -2895,6 +2896,8 @@ recheck_relation_needs_vacanalyze(Oid relid, * threshold. This threshold is calculated as * * threshold = vac_base_thresh + vac_scale_factor * reltuples + * if (threshold > vac_max_thresh) + * threshold = vac_max_thres; * * For analyze, the analysis done is that the number of tuples inserted, * deleted and updated since the last analyze exceeds a threshold calculated @@ -2933,6 +2936,7 @@ relation_needs_vacanalyze(Oid relid, /* constants from reloptions or GUC variables */ int vac_base_thresh, + vac_max_thresh, vac_ins_base_thresh, anl_base_thresh; float4 vac_scale_factor, @@ -2974,6 +2978,11 @@ relation_needs_vacanalyze(Oid relid, ? relopts->vacuum_threshold : autovacuum_vac_thresh; + /* -1 is used to disable max threshold */ + vac_max_thresh = (relopts && relopts->vacuum_max_threshold >= -1) + ? relopts->vacuum_max_threshold + : autovacuum_vac_max_thresh; + vac_ins_scale_factor = (relopts && relopts->vacuum_ins_scale_factor >= 0) ? relopts->vacuum_ins_scale_factor : autovacuum_vac_ins_scale; @@ -3047,6 +3056,9 @@ relation_needs_vacanalyze(Oid relid, reltuples = 0; vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples; + if (vac_max_thresh >= 0 && vacthresh > (float4) vac_max_thresh) + vacthresh = (float4) vac_max_thresh; + vacinsthresh = (float4) vac_ins_base_thresh + vac_ins_scale_factor * reltuples; anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples; diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c index c9d8cd796a8..7270abbc64a 100644 --- a/src/backend/utils/misc/guc_tables.c +++ b/src/backend/utils/misc/guc_tables.c @@ -3425,6 +3425,15 @@ struct config_int ConfigureNamesInt[] = 50, 0, INT_MAX, NULL, NULL, NULL }, + { + {"autovacuum_vacuum_max_threshold", PGC_SIGHUP, AUTOVACUUM, + gettext_noop("Maximum number of tuple updates or deletes prior to vacuum."), + NULL + }, + &autovacuum_vac_max_thresh, + 100000000, -1, INT_MAX, + NULL, NULL, NULL + }, { {"autovacuum_vacuum_insert_threshold", PGC_SIGHUP, AUTOVACUUM, gettext_noop("Minimum number of tuple inserts prior to vacuum, or -1 to disable insert vacuums."), diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index 079efa1baa7..d38112f3943 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -661,6 +661,9 @@ autovacuum_worker_slots = 16 # autovacuum worker slots to allocate #autovacuum_naptime = 1min # time between autovacuum runs #autovacuum_vacuum_threshold = 50 # min number of row updates before # vacuum +#autovacuum_vacuum_max_threshold = 100000000 # max number of row updates + # before vacuum; -1 disables max + # threshold #autovacuum_vacuum_insert_threshold = 1000 # min number of row inserts # before vacuum; -1 disables insert # vacuums diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 81cbf10aa28..5f6897c8486 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -1368,6 +1368,7 @@ static const char *const table_storage_parameters[] = { "autovacuum_vacuum_cost_limit", "autovacuum_vacuum_insert_scale_factor", "autovacuum_vacuum_insert_threshold", + "autovacuum_vacuum_max_threshold", "autovacuum_vacuum_scale_factor", "autovacuum_vacuum_threshold", "fillfactor", @@ -1384,6 +1385,7 @@ static const char *const table_storage_parameters[] = { "toast.autovacuum_vacuum_cost_limit", "toast.autovacuum_vacuum_insert_scale_factor", "toast.autovacuum_vacuum_insert_threshold", + "toast.autovacuum_vacuum_max_threshold", "toast.autovacuum_vacuum_scale_factor", "toast.autovacuum_vacuum_threshold", "toast.log_autovacuum_min_duration", diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h index 54e01c81d68..06d4a593575 100644 --- a/src/include/postmaster/autovacuum.h +++ b/src/include/postmaster/autovacuum.h @@ -33,6 +33,7 @@ extern PGDLLIMPORT int autovacuum_max_workers; extern PGDLLIMPORT int autovacuum_work_mem; extern PGDLLIMPORT int autovacuum_naptime; extern PGDLLIMPORT int autovacuum_vac_thresh; +extern PGDLLIMPORT int autovacuum_vac_max_thresh; extern PGDLLIMPORT double autovacuum_vac_scale; extern PGDLLIMPORT int autovacuum_vac_ins_thresh; extern PGDLLIMPORT double autovacuum_vac_ins_scale; diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 33d1e4a4e2e..48b95f211f3 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -309,6 +309,7 @@ typedef struct AutoVacOpts { bool enabled; int vacuum_threshold; + int vacuum_max_threshold; int vacuum_ins_threshold; int analyze_threshold; int vacuum_cost_limit; -- 2.39.5 (Apple Git-154) --YwvA4pfL6umvmJzT--