Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tuiSM-00AyYk-I4 for pgsql-hackers@arkaria.postgresql.org; Wed, 19 Mar 2025 01:42:54 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1tuiSL-009iHO-0Z for pgsql-hackers@arkaria.postgresql.org; Wed, 19 Mar 2025 01:42:53 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tuiSK-009iD7-J8 for pgsql-hackers@lists.postgresql.org; Wed, 19 Mar 2025 01:42:52 +0000 Received: from mail-yw1-x1131.google.com ([2607:f8b0:4864:20::1131]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1tuiSG-003fWp-2F for pgsql-hackers@postgresql.org; Wed, 19 Mar 2025 01:42:52 +0000 Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-6f768e9be1aso2396077b3.0 for ; Tue, 18 Mar 2025 18:42:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1742348568; x=1742953368; darn=postgresql.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=HeUalMY/zm6m9pEmrMJtuWaT1wOl4t+ONX9bAQgURbE=; b=eD0RBilO1HV5OnPbO+YeRl/gZHXFT5gaDYJTtA4YHVabDtlrmXD2ph5WfpVrLq9er4 linmPBfFEHB/GTXuZ9dkMKidIB9plaXplNkevv4Hd1EtjfRQg5ncaPtv7R0G5G3ZeRBD 4UqDd3GB62iH5AzVM0ikpiVnr43qVUvXBsyRHIBo3XSvYAPpepjGwogXBGvqBIsZrjg1 JjAYwxjAquA176/0KeOUiJH2I/Wz+QM7tUwgOXN5pbu5+I6/j4pkLL8DI1fXtMFKOWCG P0SGP6H9dLhpH55svS1amTPvcRHpJ1qRRfIp+ujle1magt+JNVF55Lxp9z8vg9vJnZvh 8iXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742348568; x=1742953368; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=HeUalMY/zm6m9pEmrMJtuWaT1wOl4t+ONX9bAQgURbE=; b=wsVUOuimIcSeNZD6IsFm2H3Nn2WrKJ4YQHKtlqYaHKVWvmkauOb46bIMZcYvt+RRqK o3pO+VWSdo+pW/9FuABjlf8QbydsKg9NEHqJumY56OdRLaBR+eObulG0RL4dVRwulcr1 ISEOotK1MGaSyGQXzUsF2gzgtUHyK0ACldM0jaeA5q2F5AfWFWVeNUsNK0ze+vXzX7X6 VepKB2fANGcWbk88CN6OewFsthmhqlfFYLJ6n1xnl8IJ80PogwuvR2+ncxVTUztAgYNw oN/zyueohBBcJu47gnsqHEGVTa4edZ3C1QlwqbkNmcpW6k055H8lzSMLTm+fazlagI/3 An6g== X-Forwarded-Encrypted: i=1; AJvYcCV7KhjW37oyr7hEW1Xex1PLzkb4DcaUka+mG5BwazEAv4+ea9YEi7crtW2yLbM/mObZFMXq43SzmqvSgrfk@postgresql.org X-Gm-Message-State: AOJu0YxvFS2kZgmzyGteL+A2JXLkX3MpZiLrK58Co4pV5P26AKQeUM40 FOcBIYTaBO0WnvWHcAWN12T3aMWLYi8am0REDphOHNdhVPnIO1u8 X-Gm-Gg: ASbGncv/K5+V/5smjMl3axWfR7737T2yRjDaeEWuPCxVOymsqwE57eiy2kfhSs3nAAz gEQDGSjGEdjyhq8/tBN/XiUZVMxOqSqKP8gi+eXUF6UAlK+ALaqgkbHl3zmwgQHcS0qaIy8UpwJ vdskkcYUKFD/CulFMGiIumN3gSUIH6QrBAjEC36V3hWFFCwjhRNDLYIT+XCT7l2LmgDCyay0Rbs jBvReI5ppyjn1uManCVVG1xt7B4L89IEDnh5DIT4pQ1dB1TKmFHBzy2vzK0gXnPoHnyAgQOdCUH M2TC3SPM02GKH0DkLen0SdqD4Q5jf0qiXIyBkejjFy9FBj1+rO2q5C5dUH5hjlBv41hPv4CGFbF vUWHVk3jsPP2LWJ0yg9wMP2Mp117klbEVLnpx0v2t X-Google-Smtp-Source: AGHT+IGkqP1ac9d7iuDtnWNQGshDdQ7Na8HZajIkluNNftFpKQp5bNIll5vG9yXalbYcLzwxoypTbg== X-Received: by 2002:a05:690c:2b0e:b0:6f9:c8d9:50f with SMTP id 00721157ae682-6fff2ee2990mr67778397b3.2.1742348567971; Tue, 18 Mar 2025 18:42:47 -0700 (PDT) Received: from nathan (162-195-168-172.lightspeed.stlsmo.sbcglobal.net. [162.195.168.172]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6ff32cb5971sm28582107b3.108.2025.03.18.18.42.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Mar 2025 18:42:47 -0700 (PDT) Date: Tue, 18 Mar 2025 20:42:45 -0500 From: Nathan Bossart To: Robert Treat Cc: Laurenz Albe , Gurjeet Singh , Andres Freund , Will Storey , Fujii Masao , Robert Haas , Postgres Hackers Subject: Re: Disabling vacuum truncate for autovacuum Message-ID: References: <28773a66-fb88-41cf-a7ec-4216e6c91c94@oss.nttdata.com> <6f2f2167f4be09e6ca9251c8f69dfe01809d68be.camel@cybertec.at> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="BtaF22Kn62uCKP1I" Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --BtaF22Kn62uCKP1I Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Mar 17, 2025 at 10:14:51AM -0500, Nathan Bossart wrote: > Thank you all for the discussion. I've attempted to address the > outstanding feedback into the new version of the patch. Here is a new version of the patch with tests and some other light edits. I feel like this is committable, but I'll wait for a couple more days for any other feedback or objections. -- nathan --BtaF22Kn62uCKP1I Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="v5-0001-Add-vacuum_truncate-configuration-parameter.patch" From daba49ce70965f657296d8e452f8d73992d54c48 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 18 Mar 2025 20:23:50 -0500 Subject: [PATCH v5 1/1] Add vacuum_truncate configuration parameter. This new parameter works just like the eponymous storage parameter: if set to true (which is the default), autovacuum and VACUUM attempt to truncate any empty pages at the end of the table. It is primarily intended to help users avoid locking issues on hot standbys. The setting can be overridden with the storage parameter or VACUUM's TRUNCATE option. Since there's presently no way to determine whether a Boolean storage parameter is explicitly set or has just picked up the default value, this commit also introduces an isset_offset member to relopt_parse_elt. Suggested-by: Will Storey Author: Nathan Bossart Co-authored-by: Gurjeet Singh Reviewed-by: Laurenz Albe Reviewed-by: Fujii Masao Reviewed-by: Robert Treat Discussion: https://postgr.es/m/Z2DE4lDX4tHqNGZt%40dev.null --- doc/src/sgml/config.sgml | 23 ++++++++++++++++ doc/src/sgml/ref/create_table.sgml | 13 +++------ doc/src/sgml/ref/vacuum.sgml | 3 ++- src/backend/access/common/reloptions.c | 14 +++++++++- src/backend/commands/vacuum.c | 17 +++++++++--- src/backend/utils/misc/guc_tables.c | 9 +++++++ src/backend/utils/misc/postgresql.conf.sample | 4 +++ src/include/access/reloptions.h | 1 + src/include/commands/vacuum.h | 1 + src/include/utils/rel.h | 1 + src/test/regress/expected/vacuum.out | 27 +++++++++++++++++++ src/test/regress/sql/vacuum.sql | 10 +++++++ 12 files changed, 108 insertions(+), 15 deletions(-) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 9e9c02cde83..c5a05a96c12 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -9504,6 +9504,29 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; + + vacuum_truncate (boolean) + + vacuum_truncate configuration parameter + + + + + Enables or disables vacuum to try to truncate off any empty pages at + the end of the table. The default value is true. + If true, VACUUM and autovacuum + do the truncation and the disk space for the truncated pages is + returned to the operating system. Note that the truncation requires + an ACCESS EXCLUSIVE lock on the table. The + TRUNCATE parameter of + VACUUM, if + specified, overrides the value of this parameter. The setting can + also be overridden for individual tables by changing table storage + parameters. + + + + diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index 5304b738322..e5c034d724e 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -1692,15 +1692,10 @@ WITH ( MODULUS numeric_literal, REM - Enables or disables vacuum to try to truncate off any empty pages - at the end of this table. The default value is true. - If true, VACUUM and - autovacuum do the truncation and the disk space for - the truncated pages is returned to the operating system. - Note that the truncation requires ACCESS EXCLUSIVE - lock on the table. The TRUNCATE parameter - of VACUUM, if specified, overrides the value - of this option. + Per-table value for parameter. The + TRUNCATE parameter of + VACUUM, if + specified, overrides the value of this option. diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 971b1237d47..bd5dcaf86a5 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -265,7 +265,8 @@ VACUUM [ ( option [, ...] ) ] [ vacuum_truncate + and is the default unless + is set to false or the vacuum_truncate option has been set to false for the table to be vacuumed. Setting this option to false may be useful to avoid ACCESS EXCLUSIVE lock on the table that diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c index 59fb53e7707..645b5c00467 100644 --- a/src/backend/access/common/reloptions.c +++ b/src/backend/access/common/reloptions.c @@ -1779,6 +1779,17 @@ fillRelOptions(void *rdopts, Size basesize, char *itempos = ((char *) rdopts) + elems[j].offset; char *string_val; + /* + * If isset_offset is provided, store whether the reloption is + * set there. + */ + if (elems[j].isset_offset > 0) + { + char *setpos = ((char *) rdopts) + elems[j].isset_offset; + + *(bool *) setpos = options[i].isset; + } + switch (options[i].gen->type) { case RELOPT_TYPE_BOOL: @@ -1901,7 +1912,7 @@ default_reloptions(Datum reloptions, bool validate, relopt_kind kind) {"vacuum_index_cleanup", RELOPT_TYPE_ENUM, offsetof(StdRdOptions, vacuum_index_cleanup)}, {"vacuum_truncate", RELOPT_TYPE_BOOL, - offsetof(StdRdOptions, vacuum_truncate)}, + offsetof(StdRdOptions, vacuum_truncate), offsetof(StdRdOptions, vacuum_truncate_set)}, {"vacuum_max_eager_freeze_failure_rate", RELOPT_TYPE_REAL, offsetof(StdRdOptions, vacuum_max_eager_freeze_failure_rate)} }; @@ -1981,6 +1992,7 @@ build_local_reloptions(local_relopts *relopts, Datum options, bool validate) elems[i].optname = opt->option->name; elems[i].opttype = opt->option->type; elems[i].offset = opt->offset; + elems[i].isset_offset = 0; /* not supported for local relopts yet */ i++; } diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index e81c9a8aba3..f0a7b87808d 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -78,6 +78,7 @@ int vacuum_failsafe_age; int vacuum_multixact_failsafe_age; double vacuum_max_eager_freeze_failure_rate; bool track_cost_delay_timing; +bool vacuum_truncate; /* * Variables for cost-based vacuum delay. The defaults differ between @@ -2198,13 +2199,21 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, ((StdRdOptions *) rel->rd_options)->vacuum_max_eager_freeze_failure_rate; /* - * Set truncate option based on truncate reloption if it wasn't specified - * in VACUUM command, or when running in an autovacuum worker + * Set truncate option based on truncate reloption or GUC if it wasn't + * specified in VACUUM command, or when running in an autovacuum worker */ if (params->truncate == VACOPTVALUE_UNSPECIFIED) { - if (rel->rd_options == NULL || - ((StdRdOptions *) rel->rd_options)->vacuum_truncate) + StdRdOptions *opts = (StdRdOptions *) rel->rd_options; + + if (opts && opts->vacuum_truncate_set) + { + if (opts->vacuum_truncate) + params->truncate = VACOPTVALUE_ENABLED; + else + params->truncate = VACOPTVALUE_DISABLED; + } + else if (vacuum_truncate) params->truncate = VACOPTVALUE_ENABLED; else params->truncate = VACOPTVALUE_DISABLED; diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c index 60a40ed445a..7da11793a1b 100644 --- a/src/backend/utils/misc/guc_tables.c +++ b/src/backend/utils/misc/guc_tables.c @@ -2130,6 +2130,15 @@ struct config_bool ConfigureNamesBool[] = NULL, NULL, NULL }, + { + {"vacuum_truncate", PGC_USERSET, VACUUM_AUTOVACUUM, + gettext_noop("Enables vacuum to truncate empty pages at the end of the table."), + }, + &vacuum_truncate, + true, + NULL, NULL, NULL + }, + /* End-of-list marker */ { {NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index beb05a89501..bc4ebbba2ae 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -722,6 +722,10 @@ autovacuum_worker_slots = 16 # autovacuum worker slots to allocate #vacuum_multixact_failsafe_age = 1600000000 #vacuum_max_eager_freeze_failure_rate = 0.03 # 0 disables eager scanning +# - Default Behavior - + +#vacuum_truncate = on # enable truncation after vacuum + #------------------------------------------------------------------------------ # CLIENT CONNECTION DEFAULTS #------------------------------------------------------------------------------ diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h index 43445cdcc6c..146aed47c2d 100644 --- a/src/include/access/reloptions.h +++ b/src/include/access/reloptions.h @@ -152,6 +152,7 @@ typedef struct const char *optname; /* option's name */ relopt_type opttype; /* option's datatype */ int offset; /* offset of field in result struct */ + int isset_offset; /* if > 0, offset of "is set" field */ } relopt_parse_elt; /* Local reloption definition */ diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h index baacc63f590..bc37a80dc74 100644 --- a/src/include/commands/vacuum.h +++ b/src/include/commands/vacuum.h @@ -304,6 +304,7 @@ extern PGDLLIMPORT int vacuum_multixact_freeze_table_age; extern PGDLLIMPORT int vacuum_failsafe_age; extern PGDLLIMPORT int vacuum_multixact_failsafe_age; extern PGDLLIMPORT bool track_cost_delay_timing; +extern PGDLLIMPORT bool vacuum_truncate; /* * Relevant for vacuums implementing eager scanning. Normal vacuums may diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index db3e504c3d2..d94fddd7cef 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -344,6 +344,7 @@ typedef struct StdRdOptions int parallel_workers; /* max number of parallel workers */ StdRdOptIndexCleanup vacuum_index_cleanup; /* controls index vacuuming */ bool vacuum_truncate; /* enables vacuum to truncate a relation */ + bool vacuum_truncate_set; /* whether vacuum_truncate is set */ /* * Fraction of pages in a relation that vacuum can eagerly scan and fail diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 3f91b69b324..0abcc99989e 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -236,6 +236,7 @@ SELECT pg_relation_size('vac_truncate_test') > 0; t (1 row) +SET vacuum_truncate = false; VACUUM (DISABLE_PAGE_SKIPPING) vac_truncate_test; SELECT pg_relation_size('vac_truncate_test') = 0; ?column? @@ -244,6 +245,32 @@ SELECT pg_relation_size('vac_truncate_test') = 0; (1 row) VACUUM (TRUNCATE FALSE, FULL TRUE) vac_truncate_test; +ALTER TABLE vac_truncate_test RESET (vacuum_truncate); +INSERT INTO vac_truncate_test VALUES (1, NULL), (NULL, NULL); +ERROR: null value in column "i" of relation "vac_truncate_test" violates not-null constraint +DETAIL: Failing row contains (null, null). +VACUUM (DISABLE_PAGE_SKIPPING) vac_truncate_test; +SELECT pg_relation_size('vac_truncate_test') > 0; + ?column? +---------- + t +(1 row) + +RESET vacuum_truncate; +VACUUM (TRUNCATE FALSE, DISABLE_PAGE_SKIPPING) vac_truncate_test; +SELECT pg_relation_size('vac_truncate_test') > 0; + ?column? +---------- + t +(1 row) + +VACUUM (DISABLE_PAGE_SKIPPING) vac_truncate_test; +SELECT pg_relation_size('vac_truncate_test') = 0; + ?column? +---------- + t +(1 row) + DROP TABLE vac_truncate_test; -- partitioned table CREATE TABLE vacparted (a int, b char) PARTITION BY LIST (a); diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql index 058add027f1..a72bdb5b619 100644 --- a/src/test/regress/sql/vacuum.sql +++ b/src/test/regress/sql/vacuum.sql @@ -194,9 +194,19 @@ CREATE TEMP TABLE vac_truncate_test(i INT NOT NULL, j text) INSERT INTO vac_truncate_test VALUES (1, NULL), (NULL, NULL); VACUUM (TRUNCATE FALSE, DISABLE_PAGE_SKIPPING) vac_truncate_test; SELECT pg_relation_size('vac_truncate_test') > 0; +SET vacuum_truncate = false; VACUUM (DISABLE_PAGE_SKIPPING) vac_truncate_test; SELECT pg_relation_size('vac_truncate_test') = 0; VACUUM (TRUNCATE FALSE, FULL TRUE) vac_truncate_test; +ALTER TABLE vac_truncate_test RESET (vacuum_truncate); +INSERT INTO vac_truncate_test VALUES (1, NULL), (NULL, NULL); +VACUUM (DISABLE_PAGE_SKIPPING) vac_truncate_test; +SELECT pg_relation_size('vac_truncate_test') > 0; +RESET vacuum_truncate; +VACUUM (TRUNCATE FALSE, DISABLE_PAGE_SKIPPING) vac_truncate_test; +SELECT pg_relation_size('vac_truncate_test') > 0; +VACUUM (DISABLE_PAGE_SKIPPING) vac_truncate_test; +SELECT pg_relation_size('vac_truncate_test') = 0; DROP TABLE vac_truncate_test; -- partitioned table -- 2.39.5 (Apple Git-154) --BtaF22Kn62uCKP1I--