Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tXQvy-006oaM-VU for pgsql-hackers@arkaria.postgresql.org; Mon, 13 Jan 2025 20:21:15 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1tXQvx-00Cvol-3Q for pgsql-hackers@arkaria.postgresql.org; Mon, 13 Jan 2025 20:21:13 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tXQvw-00Cvoc-Of for pgsql-hackers@lists.postgresql.org; Mon, 13 Jan 2025 20:21:13 +0000 Received: from mail-il1-x12e.google.com ([2607:f8b0:4864:20::12e]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1tXQvs-000Eck-2D for pgsql-hackers@lists.postgresql.org; Mon, 13 Jan 2025 20:21:12 +0000 Received: by mail-il1-x12e.google.com with SMTP id e9e14a558f8ab-3cdce23f3e7so35525155ab.0 for ; Mon, 13 Jan 2025 12:21:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736799667; x=1737404467; darn=lists.postgresql.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=zsbHehrmyTqszSq+aZFrgTYA7ZjFh/moNJmRY6/NWzw=; b=ja50p1gfVzU0lon/8AG1yNxY62z3pJ0QpUQ4BnJjzxqqX5g3B8xqKc+XnP8CGD5y2E l3NmB35lh8uS0jyW7HHP3sA1FK3lNLymOmsFdar8BrbF+QAT+gxH2yjR4qWJbSfIEhRD BGjCh0yidd//ORkh8PsiyIAPyIVFXFCNaZCAuuWuQtiHAH/0AFqe2gfnDeNyRhQLxxZ/ yN3GuXSYjj2+Wvk+JWyjgDdCWzxkB2lkMofitN7pgR1uXb2LzJ7KoeqEH+AN8ehzqEUY s8lfULR0bxlhyeKS13gMKk0sC2N61TZQpUu/jQb1RVmlEVO2y69Rdc46WnXSR1nB4JyX hWAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736799667; x=1737404467; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zsbHehrmyTqszSq+aZFrgTYA7ZjFh/moNJmRY6/NWzw=; b=Mey/+BKw7OdJuMPHrV5SZCHl+VCYZMusL4QunfXNFvtabBPMLInJnuZMCDD3XQDb5e t5bymTkNAeAM6bDT9L2jRdwKwFz9MQ9nqdDduDxEsvxzZR5VniKFPwpEBE8omxxPzR0r HDGVAi2I+wkyCOJVetfxlz2iAuIcfBtISn2OlZeGyMYyR0MmJm56mrhhVxDQs8uOP0qW PcKDfJxHAc0gT+dlp2tTWCTNQPPFkDGIAJE2JXY8qkTbN/hYs4L4nUy9ItCA7gUw9ucD w0P4YeNPtStrEiSBborNQC+ZBBD5PCn9UPXxaa+wb62BkyVllgNxYNFM+ZZFsPUlNWNC 4MFg== X-Forwarded-Encrypted: i=1; AJvYcCV0hCWjGTxRQ5mF2ZC+/x8aF9SRv2S4CQhSu8r2bIzMKPIdgBMTk3LSz5iwH5dQ3Zdib24bF86g+txB63u9@lists.postgresql.org X-Gm-Message-State: AOJu0YyxcMLfZw74lNvIYLhiUtiwh7IIOMhvPepQx6G1lWJahr5kW3mf klDwFqHoytDQ3Rab0dcCWVsJBSyuSdFT57GT/F3NvmHZqvAwDtnD X-Gm-Gg: ASbGnctDbhVrayWuBPMOhJqpptz6wsmWvEVML9AVMK+bgItW/PVe4A2RD9IxZngygWq WOSy0wOTiZsz6tVWXoOju7QUMJN41E1ZRK01abrDB8PiN/Uyc/05y5mjJdRLcTaIocmCC2xpw6e yNP/TI1NHAOLZTS/0o6/oAeW7ne/0PQqT5iOhX3G0T15DjQKmjRBUSB0FcXVQHKal8ZIHFfpMBt yjLi+9s+6By9HZwz1lwgfT/VCJqipNxWlvXK8puUvt4lcUIpvFQyUrQEoPp4mr0bg1mLHCPf9Bj 0yMftNRvTn/kwYqscd1i2gSCUexORMgZgmSNI25Dk7+mwF4= X-Google-Smtp-Source: AGHT+IFBzFegrmSIZJ7RFC3lKe0vuG/kjvgQv4B4jiRuWglqAjUEbQSML9i6br1uVueWBzpfgTxUZA== X-Received: by 2002:a05:6e02:1f02:b0:3a7:955e:1cc5 with SMTP id e9e14a558f8ab-3ce3a9a4288mr187438625ab.1.1736799667004; Mon, 13 Jan 2025 12:21:07 -0800 (PST) Received: from nathan (162-195-168-172.lightspeed.stlsmo.sbcglobal.net. [162.195.168.172]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4ea1b613118sm2965782173.36.2025.01.13.12.21.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jan 2025 12:21:06 -0800 (PST) Date: Mon, 13 Jan 2025 14:21:04 -0600 From: Nathan Bossart To: Robert Treat Cc: =?iso-8859-1?Q?Fr=E9d=E9ric?= Yhuel , wenhui qiu , Robert Haas , "Imseih (AWS), Sami" , David Rowley , Joe Conway , Michael Banck , Laurenz Albe , Melanie Plageman , PostgreSQL Hackers Subject: Re: New GUC autovacuum_max_threshold ? Message-ID: References: <2ca7f9c7-5da9-46cb-9072-39b71bfab948@dalibo.com> <93f3dab5-caa4-4213-9be0-855d5aaa19bf@dalibo.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="ryZWRvV/BreIoNMg" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --ryZWRvV/BreIoNMg Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Here is a rebased version of the patch (commit ca9c6a5 adjusted the documentation for vacuum-related GUCs). -- nathan --ryZWRvV/BreIoNMg Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: attachment; filename="v5-0001-Introduce-autovacuum_max_threshold.patch" Content-Transfer-Encoding: 8bit From d78d621233734abc54d0273526983bf64b88e7ba Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 8 Jan 2025 13:11:52 -0600 Subject: [PATCH v5 1/1] Introduce autovacuum_max_threshold. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Author: Nathan Bossart, Frédéric Yhuel Reviewed-by: Melanie Plageman, Robert Haas, Laurenz Albe, Michael Banck, Joe Conway, Sami Imseih, David Rowley, wenhui qiu Discussion: https://postgr.es/m/956435f8-3b2f-47a6-8756-8c54ded61802%40dalibo.com --- doc/src/sgml/config.sgml | 24 +++++++++++++++++++ doc/src/sgml/ref/create_table.sgml | 15 ++++++++++++ src/backend/access/common/reloptions.c | 11 +++++++++ src/backend/postmaster/autovacuum.c | 12 ++++++++++ src/backend/utils/misc/guc_tables.c | 9 +++++++ src/backend/utils/misc/postgresql.conf.sample | 2 ++ src/bin/psql/tab-complete.in.c | 2 ++ src/include/postmaster/autovacuum.h | 1 + src/include/utils/rel.h | 1 + 9 files changed, 77 insertions(+) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 3f41a17b1fe..5a0ed9ac36b 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -8579,6 +8579,30 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; + + autovacuum_max_threshold (integer) + + autovacuum_max_threshold + configuration parameter + + + + + Specifies the maximum number of updated or deleted tuples needed to + trigger a VACUUM in any one table, i.e., a cap on + the value calculated with + autovacuum_vacuum_threshold and + autovacuum_vacuum_scale_factor. The default is + 100,000,000 tuples. If -1 is specified, autovacuum will not enforce a + maximum number of updated or deleted tuples that will trigger a + VACUUM operation. This parameter can only be set + in the postgresql.conf file or on the server + command line; but the setting can be overridden for individual tables + by changing storage parameters. + + + + autovacuum_vacuum_insert_threshold (integer) diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index 2237321cb4f..155c78ad6ba 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -1712,6 +1712,21 @@ WITH ( MODULUS numeric_literal, REM + + autovacuum_max_threshold, toast.autovacuum_max_threshold (integer) + + autovacuum_max_threshold + storage parameter + + + + + Per-table value for + parameter. + + + + autovacuum_vacuum_scale_factor, toast.autovacuum_vacuum_scale_factor (floating point) diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c index e587abd9990..fbae300a128 100644 --- a/src/backend/access/common/reloptions.c +++ b/src/backend/access/common/reloptions.c @@ -231,6 +231,15 @@ static relopt_int intRelOpts[] = }, -1, 0, INT_MAX }, + { + { + "autovacuum_max_threshold", + "Maximum number of tuple updates or deletes prior to vacuum", + RELOPT_KIND_HEAP | RELOPT_KIND_TOAST, + ShareUpdateExclusiveLock + }, + -2, -1, INT_MAX + }, { { "autovacuum_vacuum_insert_threshold", @@ -1843,6 +1852,8 @@ default_reloptions(Datum reloptions, bool validate, relopt_kind kind) offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, enabled)}, {"autovacuum_vacuum_threshold", RELOPT_TYPE_INT, offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, vacuum_threshold)}, + {"autovacuum_max_threshold", RELOPT_TYPE_INT, + offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, vacuum_max_threshold)}, {"autovacuum_vacuum_insert_threshold", RELOPT_TYPE_INT, offsetof(StdRdOptions, autovacuum) + offsetof(AutoVacOpts, vacuum_ins_threshold)}, {"autovacuum_analyze_threshold", RELOPT_TYPE_INT, diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c index 0ab921a169b..ea48fba73f8 100644 --- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -120,6 +120,7 @@ int autovacuum_max_workers; int autovacuum_work_mem = -1; int autovacuum_naptime; int autovacuum_vac_thresh; +int autovacuum_max_thresh; double autovacuum_vac_scale; int autovacuum_vac_ins_thresh; double autovacuum_vac_ins_scale; @@ -2895,6 +2896,8 @@ recheck_relation_needs_vacanalyze(Oid relid, * threshold. This threshold is calculated as * * threshold = vac_base_thresh + vac_scale_factor * reltuples + * if (threshold > vac_max_thresh) + * threshold = vac_max_thres; * * For analyze, the analysis done is that the number of tuples inserted, * deleted and updated since the last analyze exceeds a threshold calculated @@ -2933,6 +2936,7 @@ relation_needs_vacanalyze(Oid relid, /* constants from reloptions or GUC variables */ int vac_base_thresh, + vac_max_thresh, vac_ins_base_thresh, anl_base_thresh; float4 vac_scale_factor, @@ -2974,6 +2978,11 @@ relation_needs_vacanalyze(Oid relid, ? relopts->vacuum_threshold : autovacuum_vac_thresh; + /* -1 is used to disable max threshold */ + vac_max_thresh = (relopts && relopts->vacuum_max_threshold >= -1) + ? relopts->vacuum_max_threshold + : autovacuum_max_thresh; + vac_ins_scale_factor = (relopts && relopts->vacuum_ins_scale_factor >= 0) ? relopts->vacuum_ins_scale_factor : autovacuum_vac_ins_scale; @@ -3047,6 +3056,9 @@ relation_needs_vacanalyze(Oid relid, reltuples = 0; vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples; + if (vac_max_thresh >= 0 && vacthresh > (float4) vac_max_thresh) + vacthresh = (float4) vac_max_thresh; + vacinsthresh = (float4) vac_ins_base_thresh + vac_ins_scale_factor * reltuples; anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples; diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c index c9d8cd796a8..2a18ba80c87 100644 --- a/src/backend/utils/misc/guc_tables.c +++ b/src/backend/utils/misc/guc_tables.c @@ -3425,6 +3425,15 @@ struct config_int ConfigureNamesInt[] = 50, 0, INT_MAX, NULL, NULL, NULL }, + { + {"autovacuum_max_threshold", PGC_SIGHUP, AUTOVACUUM, + gettext_noop("Maximum number of tuple updates or deletes prior to vacuum."), + NULL + }, + &autovacuum_max_thresh, + 100000000, -1, INT_MAX, + NULL, NULL, NULL + }, { {"autovacuum_vacuum_insert_threshold", PGC_SIGHUP, AUTOVACUUM, gettext_noop("Minimum number of tuple inserts prior to vacuum, or -1 to disable insert vacuums."), diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index b2bc43383db..d0e1c9b53c0 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -667,6 +667,8 @@ autovacuum_worker_slots = 16 # autovacuum worker slots to allocate #autovacuum_naptime = 1min # time between autovacuum runs #autovacuum_vacuum_threshold = 50 # min number of row updates before # vacuum +#autovacuum_max_threshold = 100000000 # max number of row updates before + # vacuum #autovacuum_vacuum_insert_threshold = 1000 # min number of row inserts # before vacuum; -1 disables insert # vacuums diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 81cbf10aa28..6ede5090cc0 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -1361,6 +1361,7 @@ static const char *const table_storage_parameters[] = { "autovacuum_freeze_max_age", "autovacuum_freeze_min_age", "autovacuum_freeze_table_age", + "autovacuum_max_threshold", "autovacuum_multixact_freeze_max_age", "autovacuum_multixact_freeze_min_age", "autovacuum_multixact_freeze_table_age", @@ -1377,6 +1378,7 @@ static const char *const table_storage_parameters[] = { "toast.autovacuum_freeze_max_age", "toast.autovacuum_freeze_min_age", "toast.autovacuum_freeze_table_age", + "toast.autovacuum_max_threshold", "toast.autovacuum_multixact_freeze_max_age", "toast.autovacuum_multixact_freeze_min_age", "toast.autovacuum_multixact_freeze_table_age", diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h index 54e01c81d68..b5c7b9b8abb 100644 --- a/src/include/postmaster/autovacuum.h +++ b/src/include/postmaster/autovacuum.h @@ -33,6 +33,7 @@ extern PGDLLIMPORT int autovacuum_max_workers; extern PGDLLIMPORT int autovacuum_work_mem; extern PGDLLIMPORT int autovacuum_naptime; extern PGDLLIMPORT int autovacuum_vac_thresh; +extern PGDLLIMPORT int autovacuum_max_thresh; extern PGDLLIMPORT double autovacuum_vac_scale; extern PGDLLIMPORT int autovacuum_vac_ins_thresh; extern PGDLLIMPORT double autovacuum_vac_ins_scale; diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 33d1e4a4e2e..48b95f211f3 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -309,6 +309,7 @@ typedef struct AutoVacOpts { bool enabled; int vacuum_threshold; + int vacuum_max_threshold; int vacuum_ins_threshold; int analyze_threshold; int vacuum_cost_limit; -- 2.39.5 (Apple Git-154) --ryZWRvV/BreIoNMg--