Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pE1Qf-0001XP-SJ for pgsql-hackers@arkaria.postgresql.org; Sat, 07 Jan 2023 05:07:38 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pE1Qe-0005pp-BU for pgsql-hackers@arkaria.postgresql.org; Sat, 07 Jan 2023 05:07:36 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pE1Qd-0005pg-OJ for pgsql-hackers@lists.postgresql.org; Sat, 07 Jan 2023 05:07:36 +0000 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1pE1QW-0000k9-LQ for pgsql-hackers@postgresql.org; Sat, 07 Jan 2023 05:07:34 +0000 Received: by mail-pj1-x102a.google.com with SMTP id z9-20020a17090a468900b00226b6e7aeeaso3926877pjf.1 for ; Fri, 06 Jan 2023 21:07:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jZTtdHiBtHPox+R07IrDJ6R99wTS++A3iOHm8YsdnYI=; b=MXPvSjUN+xtwVRodb+snxIYBhz2AleDhtkQzvKxVNjOTqguvxlT0yOY1AUXCAS3l2e dQk6ekoxCIB17a/SYG3h1nJ34y+5VkdBDnypkZWaysnQAzOZkGxpreXseCgHQsnPBcxy Ww4H4zlHFcKD7n4Qr1ERGqPQY5K0PmRxBo5sxGZWojKdj443R2h8jnFKdZ9J7rqLfIAW yJe8KVGYYUVo/U7kgarCIf7c1aeTmEGsjCgjUFdIzJuNdbiV8yZv7gIpW/OSc/dYUA+p D6XT5+9dHBXSb2f26mHPXSB7e4hazgF4oB0KhIOD42JcC4Eye3jXf6zdKZqIApzby/hc p1fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=jZTtdHiBtHPox+R07IrDJ6R99wTS++A3iOHm8YsdnYI=; b=MODbYVe36xG0u2MsVVdr0FjB4mOIe9udJ6C5q0I7urJ9j0Led/3rx5LAP9Ot1LBOlL 5pWWjoPqe/Lt+XZAzQO1W1MRdxr66QG9yPVb6ETeAdNYfL8JFDMNO3tJ0FajLd+WbSdh eQNvncvzTa8W9DUcP0SVbq8rEDCw12zcjCPA03roXMCGczwG0O7VLdfKs+d79sWjHaIF M0jUMK0aPfysxkgN69nALygjjnKi+Vgil5xm4L7i2l/USuwIXXvjVR29jUWOSC4RG7oX /YIMkCvUgygHNvOGasArpmY7clM/KQHhYdgttnJVO3QD01N52VPCe0AYRPk9nim2haFw UljA== X-Gm-Message-State: AFqh2kpOFIgpXSvF3hn9ZkJaEVN4YYFXeTVx+Kj8VncgmPshsSWBnb/i i0nPENcBw3rHGROwpfqfWcH4Ad0lZtI= X-Google-Smtp-Source: AMrXdXvXX83uWVAilSUEeKPIpHIyrl0fjh0wYvnS3gZXcpcalxIEFBfXx118CSJiETlZhIMkTW+GSw== X-Received: by 2002:a17:903:24b:b0:187:282d:6990 with SMTP id j11-20020a170903024b00b00187282d6990mr46885388plh.28.1673068047215; Fri, 06 Jan 2023 21:07:27 -0800 (PST) Received: from nathanxps13 ([50.47.162.83]) by smtp.gmail.com with ESMTPSA id l5-20020a170902d34500b001929568afe9sm1678779plk.306.2023.01.06.21.07.26 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jan 2023 21:07:26 -0800 (PST) Date: Fri, 6 Jan 2023 21:07:24 -0800 From: Nathan Bossart To: pgsql-hackers@postgresql.org Subject: Re: add PROCESS_MAIN to VACUUM Message-ID: <20230107050724.GA914213@nathanxps13> References: <20221230000028.GA435655@nathanxps13> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="5mCyUwZo2JvN/JJP" Content-Disposition: inline In-Reply-To: <20221230000028.GA435655@nathanxps13> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --5mCyUwZo2JvN/JJP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline rebased for cfbot -- Nathan Bossart Amazon Web Services: https://aws.amazon.com --5mCyUwZo2JvN/JJP Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2-0001-add-PROCESS_MAIN-to-VACUUM.patch" From 7e72b0a9f06fdfa00d5320d4c3303e67788878aa Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 29 Dec 2022 15:31:49 -0800 Subject: [PATCH v2 1/1] add PROCESS_MAIN to VACUUM --- doc/src/sgml/ref/vacuum.sgml | 13 +++++++++++++ doc/src/sgml/ref/vacuumdb.sgml | 15 +++++++++++++++ src/backend/commands/vacuum.c | 27 +++++++++++++++++++++------ src/backend/postmaster/autovacuum.c | 4 +++- src/bin/psql/tab-complete.c | 4 ++-- src/bin/scripts/t/100_vacuumdb.pl | 7 +++++++ src/bin/scripts/vacuumdb.c | 24 ++++++++++++++++++++++++ src/include/commands/vacuum.h | 1 + src/test/regress/expected/vacuum.out | 4 ++++ src/test/regress/sql/vacuum.sql | 5 +++++ 10 files changed, 95 insertions(+), 9 deletions(-) diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 8fa8421847..4266f0c985 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -33,6 +33,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ boolean ] SKIP_LOCKED [ boolean ] INDEX_CLEANUP { AUTO | ON | OFF } + PROCESS_MAIN [ boolean ] PROCESS_TOAST [ boolean ] TRUNCATE [ boolean ] PARALLEL integer @@ -238,6 +239,18 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ defname, "process_toast") == 0) process_toast = defGetBoolean(opt); else if (strcmp(opt->defname, "truncate") == 0) @@ -224,7 +227,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel) (disable_page_skipping ? VACOPT_DISABLE_PAGE_SKIPPING : 0) | (process_toast ? VACOPT_PROCESS_TOAST : 0) | (skip_database_stats ? VACOPT_SKIP_DATABASE_STATS : 0) | - (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0); + (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0) | + (process_main ? VACOPT_PROCESS_MAIN : 0); /* sanity checks on options */ Assert(params.options & (VACOPT_VACUUM | VACOPT_ANALYZE)); @@ -365,9 +369,10 @@ vacuum(List *relations, VacuumParams *params, ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("ONLY_DATABASE_STATS cannot be specified with a list of tables"))); - /* don't require people to turn off PROCESS_TOAST explicitly */ + /* don't require people to turn off PROCESS_TOAST/MAIN explicitly */ if (params->options & ~(VACOPT_VACUUM | VACOPT_VERBOSE | + VACOPT_PROCESS_MAIN | VACOPT_PROCESS_TOAST | VACOPT_ONLY_DATABASE_STATS)) ereport(ERROR, @@ -2026,10 +2031,12 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) /* * Remember the relation's TOAST relation for later, if the caller asked * us to process it. In VACUUM FULL, though, the toast table is - * automatically rebuilt by cluster_rel so we shouldn't recurse to it. + * automatically rebuilt by cluster_rel so we shouldn't recurse to it + * unless PROCESS_MAIN is disabled. */ if ((params->options & VACOPT_PROCESS_TOAST) != 0 && - (params->options & VACOPT_FULL) == 0) + ((params->options & VACOPT_FULL) == 0 || + (params->options & VACOPT_PROCESS_MAIN) == 0)) toast_relid = rel->rd_rel->reltoastrelid; else toast_relid = InvalidOid; @@ -2048,7 +2055,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) /* * Do the actual work --- either FULL or "lazy" vacuum */ - if (params->options & VACOPT_FULL) + if (params->options & VACOPT_FULL && + params->options & VACOPT_PROCESS_MAIN) { ClusterParams cluster_params = {0}; @@ -2062,7 +2070,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */ cluster_rel(relid, InvalidOid, &cluster_params); } - else + else if (params->options & VACOPT_PROCESS_MAIN) table_relation_vacuum(rel, params, vac_strategy); /* Roll back any GUC changes executed by index functions */ @@ -2089,7 +2097,14 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) * totally unimportant for toast relations. */ if (toast_relid != InvalidOid) + { + /* we force VACOPT_PROCESS_MAIN so vacuum_rel() processes it */ + bool force_opt = ((params->options & VACOPT_PROCESS_MAIN) == 0); + + params->options |= force_opt ? VACOPT_PROCESS_MAIN : 0; vacuum_rel(toast_relid, NULL, params); + params->options &= force_opt ? ~VACOPT_PROCESS_MAIN : ~0; + } /* * Now release the session-level lock on the main table. diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c index f5ea381c53..12dcb2b762 100644 --- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -2860,7 +2860,9 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map, * skip vac_update_datfrozenxid(); we'll do that separately. */ tab->at_params.options = - (dovacuum ? (VACOPT_VACUUM | VACOPT_SKIP_DATABASE_STATS) : 0) | + (dovacuum ? (VACOPT_VACUUM | + VACOPT_PROCESS_MAIN | + VACOPT_SKIP_DATABASE_STATS) : 0) | (doanalyze ? VACOPT_ANALYZE : 0) | (!wraparound ? VACOPT_SKIP_LOCKED : 0); diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c index 23750ea5fb..ef7bcc2a79 100644 --- a/src/bin/psql/tab-complete.c +++ b/src/bin/psql/tab-complete.c @@ -4597,10 +4597,10 @@ psql_completion(const char *text, int start, int end) if (ends_with(prev_wd, '(') || ends_with(prev_wd, ',')) COMPLETE_WITH("FULL", "FREEZE", "ANALYZE", "VERBOSE", "DISABLE_PAGE_SKIPPING", "SKIP_LOCKED", - "INDEX_CLEANUP", "PROCESS_TOAST", + "INDEX_CLEANUP", "PROCESS_MAIN", "PROCESS_TOAST", "TRUNCATE", "PARALLEL", "SKIP_DATABASE_STATS", "ONLY_DATABASE_STATS"); - else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS")) + else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_MAIN|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS")) COMPLETE_WITH("ON", "OFF"); else if (TailMatches("INDEX_CLEANUP")) COMPLETE_WITH("AUTO", "ON", "OFF"); diff --git a/src/bin/scripts/t/100_vacuumdb.pl b/src/bin/scripts/t/100_vacuumdb.pl index 3cfbaaec0d..46101899ae 100644 --- a/src/bin/scripts/t/100_vacuumdb.pl +++ b/src/bin/scripts/t/100_vacuumdb.pl @@ -65,6 +65,13 @@ $node->issues_sql_like( $node->command_fails( [ 'vacuumdb', '--analyze-only', '--no-truncate', 'postgres' ], '--analyze-only and --no-truncate specified together'); +$node->issues_sql_like( + [ 'vacuumdb', '--no-process-main', 'postgres' ], + qr/statement: VACUUM \(PROCESS_MAIN FALSE, SKIP_DATABASE_STATS\).*;/, + 'vacuumdb --no-process-main'); +$node->command_fails( + [ 'vacuumdb', '--analyze-only', '--no-process-main', 'postgres' ], + '--analyze-only and --no-process_main specified together'); $node->issues_sql_like( [ 'vacuumdb', '--no-process-toast', 'postgres' ], qr/statement: VACUUM \(PROCESS_TOAST FALSE, SKIP_DATABASE_STATS\).*;/, diff --git a/src/bin/scripts/vacuumdb.c b/src/bin/scripts/vacuumdb.c index 58b894216b..39be265b5b 100644 --- a/src/bin/scripts/vacuumdb.c +++ b/src/bin/scripts/vacuumdb.c @@ -43,6 +43,7 @@ typedef struct vacuumingOptions bool no_index_cleanup; bool force_index_cleanup; bool do_truncate; + bool process_main; bool process_toast; bool skip_database_stats; } vacuumingOptions; @@ -121,6 +122,7 @@ main(int argc, char *argv[]) {"force-index-cleanup", no_argument, NULL, 9}, {"no-truncate", no_argument, NULL, 10}, {"no-process-toast", no_argument, NULL, 11}, + {"no-process-main", no_argument, NULL, 12}, {NULL, 0, NULL, 0} }; @@ -148,6 +150,7 @@ main(int argc, char *argv[]) vacopts.no_index_cleanup = false; vacopts.force_index_cleanup = false; vacopts.do_truncate = true; + vacopts.process_main = true; vacopts.process_toast = true; pg_logging_init(argv[0]); @@ -260,6 +263,9 @@ main(int argc, char *argv[]) case 11: vacopts.process_toast = false; break; + case 12: + vacopts.process_main = false; + break; default: /* getopt_long already emitted a complaint */ pg_log_error_hint("Try \"%s --help\" for more information.", progname); @@ -312,6 +318,9 @@ main(int argc, char *argv[]) if (!vacopts.do_truncate) pg_fatal("cannot use the \"%s\" option when performing only analyze", "no-truncate"); + if (!vacopts.process_main) + pg_fatal("cannot use the \"%s\" option when performing only analyze", + "no-process-main"); if (!vacopts.process_toast) pg_fatal("cannot use the \"%s\" option when performing only analyze", "no-process-toast"); @@ -508,6 +517,13 @@ vacuum_one_database(ConnParams *cparams, "no-truncate", "12"); } + if (!vacopts->process_main && PQserverVersion(conn) < 160000) + { + PQfinish(conn); + pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s", + "no-process-main", "16"); + } + if (!vacopts->process_toast && PQserverVersion(conn) < 140000) { PQfinish(conn); @@ -976,6 +992,13 @@ prepare_vacuum_command(PQExpBuffer sql, int serverVersion, appendPQExpBuffer(sql, "%sTRUNCATE FALSE", sep); sep = comma; } + if (!vacopts->process_main) + { + /* PROCESS_MAIN is supported since v16 */ + Assert(serverVersion >= 160000); + appendPQExpBuffer(sql, "%sPROCESS_MAIN FALSE", sep); + sep = comma; + } if (!vacopts->process_toast) { /* PROCESS_TOAST is supported since v14 */ @@ -1090,6 +1113,7 @@ help(const char *progname) printf(_(" --min-mxid-age=MXID_AGE minimum multixact ID age of tables to vacuum\n")); printf(_(" --min-xid-age=XID_AGE minimum transaction ID age of tables to vacuum\n")); printf(_(" --no-index-cleanup don't remove index entries that point to dead tuples\n")); + printf(_(" --no-process-main skip the main relation\n")); printf(_(" --no-process-toast skip the TOAST table associated with the table to vacuum\n")); printf(_(" --no-truncate don't truncate empty pages at the end of the table\n")); printf(_(" -n, --schema=PATTERN vacuum tables in the specified schema(s) only\n")); diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h index 689dbb7702..29f5c1af04 100644 --- a/src/include/commands/vacuum.h +++ b/src/include/commands/vacuum.h @@ -190,6 +190,7 @@ typedef struct VacAttrStats #define VACOPT_DISABLE_PAGE_SKIPPING 0x80 /* don't skip any pages */ #define VACOPT_SKIP_DATABASE_STATS 0x100 /* skip vac_update_datfrozenxid() */ #define VACOPT_ONLY_DATABASE_STATS 0x200 /* only vac_update_datfrozenxid() */ +#define VACOPT_PROCESS_MAIN 0x400 /* process main relation */ /* * Values used by index_cleanup and truncate params. diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index d860be0e20..e93594ad2a 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -288,6 +288,10 @@ VACUUM (SKIP_DATABASE_STATS) vactst; VACUUM (ONLY_DATABASE_STATS); VACUUM (ONLY_DATABASE_STATS) vactst; -- error ERROR: ONLY_DATABASE_STATS cannot be specified with a list of tables +-- PROCESS_MAIN option +VACUUM (PROCESS_MAIN FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, FULL) vactst; DROP TABLE vaccluster; DROP TABLE vactst; DROP TABLE vacparted; diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql index 9da8f3e830..aaa97aa8ba 100644 --- a/src/test/regress/sql/vacuum.sql +++ b/src/test/regress/sql/vacuum.sql @@ -244,6 +244,11 @@ VACUUM (SKIP_DATABASE_STATS) vactst; VACUUM (ONLY_DATABASE_STATS); VACUUM (ONLY_DATABASE_STATS) vactst; -- error +-- PROCESS_MAIN option +VACUUM (PROCESS_MAIN FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, FULL) vactst; + DROP TABLE vaccluster; DROP TABLE vactst; DROP TABLE vacparted; -- 2.25.1 --5mCyUwZo2JvN/JJP--