Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tuCBB-004wr4-4m for pgsql-hackers@arkaria.postgresql.org; Mon, 17 Mar 2025 15:15:01 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1tuCB7-00EJjL-Ty for pgsql-hackers@arkaria.postgresql.org; Mon, 17 Mar 2025 15:14:57 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tuCB7-00EJjC-Bg for pgsql-hackers@lists.postgresql.org; Mon, 17 Mar 2025 15:14:57 +0000 Received: from mail-yw1-x1136.google.com ([2607:f8b0:4864:20::1136]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1tuCB4-003LA2-2o for pgsql-hackers@postgresql.org; Mon, 17 Mar 2025 15:14:56 +0000 Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-6fedefb1c9cso37523907b3.0 for ; Mon, 17 Mar 2025 08:14:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1742224494; x=1742829294; darn=postgresql.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Fxa2dofO62rYirZZBTTz0sp8NOcHpwLkDSwVDbWu8vI=; b=KWwhlUENrCEUuppXI/4WmY19q9I9G1f+UQmrWKPwwk97clWWYvBZ/1CEPDWigbbrAp ljeFcer4VVQPa1E3n/XnXR+lchcAtuvY9MH/vr3WLx7rOXxv3sEK7Bf1H57TicnM9QsE 2FuYpspwAWQL/ESarFRQRSNMtj5K2KYET/+aOLxTzMcn6RFrLPUc3Nu/ss5PtzY4VXUp KAYFbdEu9Dk/aFoAtT1ZySTeM27AZcWdHmr8QcS9E9uI0vmiXEq0Xbm5Q3EuWw0XT6th uLPUYEVpZYtB/eEJkf5MOR1WFcU+gDCAt0OhEACeckq+crGGtzASO085MZHj+Ztfx13U zDfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742224494; x=1742829294; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Fxa2dofO62rYirZZBTTz0sp8NOcHpwLkDSwVDbWu8vI=; b=IfTYJTvW+Xhm8P07Miwp86BxL2EoX34YVb3dY4lVpmiI0e7xqZJZdQBiXyrufkW0FH pC++3uK+tW5u3XQnRxY69vj1foXqX8krX1LK2oUD5UGfH+XQeVXZIBu1Mt/S772Tdl82 MHgdKR5x7gUgZj87grgMHas61TIb4ENoVMMfSm8FCbU/7PoIBdCrXeWe0dUDwog4wgYo 4xpvMpXipG6YK3uMgYCplETYLHjkFILOLzQdGMju3u39g4m4jTc8SKiVAYO37SaedUI1 HjoLXYHEYvZ95tda1E27YaE5LG9LB3m8uEYVli3WZAzm1fn9EVGOZOSJ5YfTtnJ31H7A zmMQ== X-Forwarded-Encrypted: i=1; AJvYcCXBBD4sKfIkeZvjIT2A42XKSibkeyJqGA1cpQlNXsMSmojBxLytlaSJqAlLKaaF4cDVfuqK+acMdg+M2bSA@postgresql.org X-Gm-Message-State: AOJu0YxeJ7h0Sl4HOaFxGGdx27wPWmtTJ2pfLP4M8q0ZIgN8O+MGkCqs 1t1nchYlFkdmNK3qw4I9M3kU8hbd6Xta1kS5q0+wBhVCtJl2YHkh X-Gm-Gg: ASbGncsJCO3B1sHstNloxeNkRMBPu46oqrqDo50bP8mDz2qFlWpRkOHdpFiMG3Cg0Rl NqebZx0GRxP9U80g+m78kAbWIHbCXTuUUjNzKDpSUYd1DEwZwE1cTv0GwH+DyyYXDCTAHZ34f1M LmX9tbj3QkrM7cb3SHpP60SjihxRFPGc5iNIEjtml2p+DBYgsYPC7Y04iInQen+ob7t3AAasalK iZqWO39XmvtbsBc7ul+c4MCcMJl0LF3iWyqPFnbxKeyaq2GvcYYDPyjcJrQ8bmYXBrGfVJ0fnb6 /uiJMkgHSlK6OYSy/NSbKHTZkkze4jdniPHPhhnIKYeMMPNgbDjpJ9p1mN0JMWCSvWEJ2iAV1Nh 2LLXH8yLfsvis7l7iHvN9nTZaJExVgLCD1UMrEMif X-Google-Smtp-Source: AGHT+IFC7R68Hpc2oNlIot4Q8vzKb+EVqVdhwlH0HPKxJRExhFsRQ7tdk8g4Otrcwi+3dJEg5hoqDg== X-Received: by 2002:a05:690c:6009:b0:6ef:9e74:c09b with SMTP id 00721157ae682-6ff460dac75mr167808677b3.33.1742224494176; Mon, 17 Mar 2025 08:14:54 -0700 (PDT) Received: from nathan (162-195-168-172.lightspeed.stlsmo.sbcglobal.net. [162.195.168.172]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6ff3288bdfbsm21967947b3.52.2025.03.17.08.14.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Mar 2025 08:14:53 -0700 (PDT) Date: Mon, 17 Mar 2025 10:14:51 -0500 From: Nathan Bossart To: Robert Treat Cc: Laurenz Albe , Gurjeet Singh , Andres Freund , Will Storey , Fujii Masao , Robert Haas , Postgres Hackers Subject: Re: Disabling vacuum truncate for autovacuum Message-ID: References: <28773a66-fb88-41cf-a7ec-4216e6c91c94@oss.nttdata.com> <6f2f2167f4be09e6ca9251c8f69dfe01809d68be.camel@cybertec.at> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="RNDgGzXBgkhvU3Qa" Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --RNDgGzXBgkhvU3Qa Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sun, Mar 16, 2025 at 06:24:59PM -0400, Robert Treat wrote: > So if the general idea is a guc "vacuum_truncate" which sets a global > default for whether vacuums and autovacuums should do truncation, and > we then have the storage parameter which would override the global > setting. And to be clear, there is also the decision on whether the > VACUUM commands default should default to truncate=on (like the > existing behavior) or truncate == vacuum_truncate guc, unless > explicitly set. I think the latter is probably the right way to go. Thank you all for the discussion. I've attempted to address the outstanding feedback into the new version of the patch. -- nathan --RNDgGzXBgkhvU3Qa Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="v4-0001-Add-vacuum_truncate-configuration-parameter.patch" From 8d367eb19f4adf01410d5f6b96ee36f15fb345ad Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 17 Mar 2025 10:11:56 -0500 Subject: [PATCH v4 1/1] Add vacuum_truncate configuration parameter. This new parameter works just like the eponymous storage parameter: if set to true (which is the default), autovacuum and VACUUM attempt to truncate any empty pages at the end of the table. The setting can be overridden for individual tables with the storage parameter or VACUUM's TRUNCATE option. It is primarily intended to help users avoid locking issues on hot standbys. Since there's presently no way to determine whether a Boolean storage parameter is explicitly set or has just picked up the default value, this commit introduces an optional isset_offset member to relopt_parse_elt. Suggested-by: Will Storey Author: Nathan Bossart Co-authored-by: Gurjeet Singh Reviewed-by: Laurenz Albe Reviewed-by: Robert Haas Reviewed-by: Fujii Masao Reviewed-by: Robert Treat Discussion: https://postgr.es/m/Z2DE4lDX4tHqNGZt%40dev.null --- doc/src/sgml/config.sgml | 22 +++++++++++++++++++ doc/src/sgml/ref/create_table.sgml | 13 ++++------- doc/src/sgml/ref/vacuum.sgml | 3 ++- src/backend/access/common/reloptions.c | 14 +++++++++++- src/backend/commands/vacuum.c | 17 ++++++++++---- src/backend/utils/misc/guc_tables.c | 9 ++++++++ src/backend/utils/misc/postgresql.conf.sample | 4 ++++ src/include/access/reloptions.h | 1 + src/include/commands/vacuum.h | 1 + src/include/utils/rel.h | 1 + 10 files changed, 70 insertions(+), 15 deletions(-) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 3d62c8bd274..18c11e284e4 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -9399,6 +9399,28 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; + + vacuum_truncate (boolean) + + vacuum_truncate configuration parameter + + + + + Enables or disables vacuum to try to truncate off any empty pages at + the end of the table. The default value is true. + If true, VACUUM and autovacuum + do the truncation and the disk space for the truncated pages is + returned to the operating system. Note that the truncation requires + an ACCESS EXCLUSIVE lock on the table. The + TRUNCATE parameter of + VACUUM, if + specified, overrides the value of this parameter. The setting can be + overridden for individual tables by changing table storage parameters. + + + + diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index 5304b738322..e5c034d724e 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -1692,15 +1692,10 @@ WITH ( MODULUS numeric_literal, REM - Enables or disables vacuum to try to truncate off any empty pages - at the end of this table. The default value is true. - If true, VACUUM and - autovacuum do the truncation and the disk space for - the truncated pages is returned to the operating system. - Note that the truncation requires ACCESS EXCLUSIVE - lock on the table. The TRUNCATE parameter - of VACUUM, if specified, overrides the value - of this option. + Per-table value for parameter. The + TRUNCATE parameter of + VACUUM, if + specified, overrides the value of this option. diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 971b1237d47..bd5dcaf86a5 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -265,7 +265,8 @@ VACUUM [ ( option [, ...] ) ] [ vacuum_truncate + and is the default unless + is set to false or the vacuum_truncate option has been set to false for the table to be vacuumed. Setting this option to false may be useful to avoid ACCESS EXCLUSIVE lock on the table that diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c index 59fb53e7707..ccbc673b228 100644 --- a/src/backend/access/common/reloptions.c +++ b/src/backend/access/common/reloptions.c @@ -1779,6 +1779,17 @@ fillRelOptions(void *rdopts, Size basesize, char *itempos = ((char *) rdopts) + elems[j].offset; char *string_val; + /* + * If isset_offset is provided, store whether the reloption is + * set there. + */ + if (elems[j].isset_offset > 0) + { + char *setpos = ((char *) rdopts) + elems[j].isset_offset; + + *(bool *) setpos = options[i].isset; + } + switch (options[i].gen->type) { case RELOPT_TYPE_BOOL: @@ -1901,7 +1912,7 @@ default_reloptions(Datum reloptions, bool validate, relopt_kind kind) {"vacuum_index_cleanup", RELOPT_TYPE_ENUM, offsetof(StdRdOptions, vacuum_index_cleanup)}, {"vacuum_truncate", RELOPT_TYPE_BOOL, - offsetof(StdRdOptions, vacuum_truncate)}, + offsetof(StdRdOptions, vacuum_truncate), offsetof(StdRdOptions, vacuum_truncate_set)}, {"vacuum_max_eager_freeze_failure_rate", RELOPT_TYPE_REAL, offsetof(StdRdOptions, vacuum_max_eager_freeze_failure_rate)} }; @@ -1981,6 +1992,7 @@ build_local_reloptions(local_relopts *relopts, Datum options, bool validate) elems[i].optname = opt->option->name; elems[i].opttype = opt->option->type; elems[i].offset = opt->offset; + elems[i].isset_offset = 0; /* not supported for local reloptions */ i++; } diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index e81c9a8aba3..f0a7b87808d 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -78,6 +78,7 @@ int vacuum_failsafe_age; int vacuum_multixact_failsafe_age; double vacuum_max_eager_freeze_failure_rate; bool track_cost_delay_timing; +bool vacuum_truncate; /* * Variables for cost-based vacuum delay. The defaults differ between @@ -2198,13 +2199,21 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, ((StdRdOptions *) rel->rd_options)->vacuum_max_eager_freeze_failure_rate; /* - * Set truncate option based on truncate reloption if it wasn't specified - * in VACUUM command, or when running in an autovacuum worker + * Set truncate option based on truncate reloption or GUC if it wasn't + * specified in VACUUM command, or when running in an autovacuum worker */ if (params->truncate == VACOPTVALUE_UNSPECIFIED) { - if (rel->rd_options == NULL || - ((StdRdOptions *) rel->rd_options)->vacuum_truncate) + StdRdOptions *opts = (StdRdOptions *) rel->rd_options; + + if (opts && opts->vacuum_truncate_set) + { + if (opts->vacuum_truncate) + params->truncate = VACOPTVALUE_ENABLED; + else + params->truncate = VACOPTVALUE_DISABLED; + } + else if (vacuum_truncate) params->truncate = VACOPTVALUE_ENABLED; else params->truncate = VACOPTVALUE_DISABLED; diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c index 9c0b10ad4dc..56dcda9546c 100644 --- a/src/backend/utils/misc/guc_tables.c +++ b/src/backend/utils/misc/guc_tables.c @@ -2118,6 +2118,15 @@ struct config_bool ConfigureNamesBool[] = NULL, NULL, NULL }, + { + {"vacuum_truncate", PGC_USERSET, VACUUM_AUTOVACUUM, + gettext_noop("Enables vacuum to truncate empty pages at the end of the table."), + }, + &vacuum_truncate, + true, + NULL, NULL, NULL + }, + /* End-of-list marker */ { {NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index 8de86e0c945..103894f3a55 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -714,6 +714,10 @@ autovacuum_worker_slots = 16 # autovacuum worker slots to allocate #vacuum_multixact_failsafe_age = 1600000000 #vacuum_max_eager_freeze_failure_rate = 0.03 # 0 disables eager scanning +# - Default Behavior - + +#vacuum_truncate = on # enable truncation after vacuum + #------------------------------------------------------------------------------ # CLIENT CONNECTION DEFAULTS #------------------------------------------------------------------------------ diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h index 43445cdcc6c..61e58a5daa3 100644 --- a/src/include/access/reloptions.h +++ b/src/include/access/reloptions.h @@ -152,6 +152,7 @@ typedef struct const char *optname; /* option's name */ relopt_type opttype; /* option's datatype */ int offset; /* offset of field in result struct */ + int isset_offset; /* if > 0, offset of "set" field */ } relopt_parse_elt; /* Local reloption definition */ diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h index baacc63f590..bc37a80dc74 100644 --- a/src/include/commands/vacuum.h +++ b/src/include/commands/vacuum.h @@ -304,6 +304,7 @@ extern PGDLLIMPORT int vacuum_multixact_freeze_table_age; extern PGDLLIMPORT int vacuum_failsafe_age; extern PGDLLIMPORT int vacuum_multixact_failsafe_age; extern PGDLLIMPORT bool track_cost_delay_timing; +extern PGDLLIMPORT bool vacuum_truncate; /* * Relevant for vacuums implementing eager scanning. Normal vacuums may diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index db3e504c3d2..d94fddd7cef 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -344,6 +344,7 @@ typedef struct StdRdOptions int parallel_workers; /* max number of parallel workers */ StdRdOptIndexCleanup vacuum_index_cleanup; /* controls index vacuuming */ bool vacuum_truncate; /* enables vacuum to truncate a relation */ + bool vacuum_truncate_set; /* whether vacuum_truncate is set */ /* * Fraction of pages in a relation that vacuum can eagerly scan and fail -- 2.39.5 (Apple Git-154) --RNDgGzXBgkhvU3Qa--