Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pIaGq-0006tm-RE for pgsql-hackers@arkaria.postgresql.org; Thu, 19 Jan 2023 19:08:21 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pIaGo-0006DR-PU for pgsql-hackers@arkaria.postgresql.org; Thu, 19 Jan 2023 19:08:18 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pIaGo-0006Ct-66 for pgsql-hackers@lists.postgresql.org; Thu, 19 Jan 2023 19:08:18 +0000 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1pIaGg-0006Oh-Te for pgsql-hackers@postgresql.org; Thu, 19 Jan 2023 19:08:17 +0000 Received: by mail-pg1-x52e.google.com with SMTP id f3so2322961pgc.2 for ; Thu, 19 Jan 2023 11:08:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Y9ClBeLTk9qSoPEGbRs25p9912jBZnp01ZJZZnI6mvk=; b=Us+4F+h8FEJuikmtfk7BhOPit5Hbz0k7nGmvxBF7TXqrzM2AyQtgJttUAJzfKvaPdM bYsek8nwWxq5WXrN/LenJWvnl26L8x/EEC+mfz4FKHtXDoDevvO+CgcLS2GJENh6rhZ5 ppRmuvjVLjg4jxQzLlA2UeahSJH7ogUE+JoljmQ8Ro+xI9tLfcqhEBpv01XsjZfNxJOR FpInQ7HLhuaZQw1lcTw1NkK/2hJV8HCVoLW3b0dQYw+OAUw9wnxIUwAtVULh9Gpt6Zj9 2NPYjb0faOrKXTPz70oyOztIZCD+XutvUdH/ltP8YK+yZ1zMDqTRw4/xZoRvK9y2d0D3 4CEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Y9ClBeLTk9qSoPEGbRs25p9912jBZnp01ZJZZnI6mvk=; b=P+f4dc2Rjpcg4Ikoc3xaqvXuUCBTnXzkwtxQsffYLfC/gD13kMQKDMBKZtzKBopY/H raztGQDC8sFklXGWx2DjeptEWUk4PjQ0GaIuXMbVPaKpasq3OrsM5jPJmuH9puF29tg0 yE1FIGFSNp8egc/s3c3y6nvz96Gz0jLQF/IpVrY7Inw3SVMVXaCKqyrAABBHJNRjNmgJ e+pjAzrk9zBFPcV1U3acJzHkOY6SDdYbI+7XxVni7MM6vuK8JBZIE7lBLqCEL3bbetWz e2KEizInjcphcErNlMalDF8jvBzNepXsLE8uyq6aiR7D9aBs28f8Q5kqrByI7soN7YMw 8iSg== X-Gm-Message-State: AFqh2kqV5b6edYKo09mu8hG64YrS7My1HteRKcrMc/CeXXkuRF5Uxwcf psGvW6TtyRvDI7rr71svDlo= X-Google-Smtp-Source: AMrXdXu0flj+mT0wjUdhXyaPjg2GOndjbX3l38ZRfAhYPPjUxUVMWpSdHDD2VX1BF3E//qF6bVy6dA== X-Received: by 2002:aa7:88d5:0:b0:58d:a683:bb2d with SMTP id k21-20020aa788d5000000b0058da683bb2dmr16053228pff.30.1674155289661; Thu, 19 Jan 2023 11:08:09 -0800 (PST) Received: from nathanxps13 ([50.47.162.83]) by smtp.gmail.com with ESMTPSA id e9-20020aa79809000000b0058d9058fe8asm9363477pfl.103.2023.01.19.11.08.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Jan 2023 11:08:08 -0800 (PST) Date: Thu, 19 Jan 2023 11:08:07 -0800 From: Nathan Bossart To: vignesh C Cc: pgsql-hackers@postgresql.org Subject: Re: add PROCESS_MAIN to VACUUM Message-ID: <20230119190807.GA4062411@nathanxps13> References: <20221230000028.GA435655@nathanxps13> <20230107050724.GA914213@nathanxps13> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="4Ckj6UjgE2iN1+kY" Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --4Ckj6UjgE2iN1+kY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Jan 19, 2023 at 05:28:25PM +0530, vignesh C wrote: > The patch does not apply on top of HEAD as in [1], please post a rebased patch: rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com --4Ckj6UjgE2iN1+kY Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0001-add-PROCESS_MAIN-to-VACUUM.patch" From 7a7f96bf4eea5be6cc252dda6bc330e77a6a3316 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 29 Dec 2022 15:31:49 -0800 Subject: [PATCH v3 1/1] add PROCESS_MAIN to VACUUM --- doc/src/sgml/ref/vacuum.sgml | 13 +++++++++++++ doc/src/sgml/ref/vacuumdb.sgml | 15 +++++++++++++++ src/backend/commands/vacuum.c | 28 ++++++++++++++++++++++------ src/backend/postmaster/autovacuum.c | 4 +++- src/bin/psql/tab-complete.c | 4 ++-- src/bin/scripts/t/100_vacuumdb.pl | 7 +++++++ src/bin/scripts/vacuumdb.c | 24 ++++++++++++++++++++++++ src/include/commands/vacuum.h | 1 + src/test/regress/expected/vacuum.out | 4 ++++ src/test/regress/sql/vacuum.sql | 5 +++++ 10 files changed, 96 insertions(+), 9 deletions(-) diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 545b23b54f..b6d30b5764 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -33,6 +33,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ boolean ] SKIP_LOCKED [ boolean ] INDEX_CLEANUP { AUTO | ON | OFF } + PROCESS_MAIN [ boolean ] PROCESS_TOAST [ boolean ] TRUNCATE [ boolean ] PARALLEL integer @@ -238,6 +239,18 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ defname, "process_toast") == 0) process_toast = defGetBoolean(opt); else if (strcmp(opt->defname, "truncate") == 0) @@ -226,7 +229,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel) (disable_page_skipping ? VACOPT_DISABLE_PAGE_SKIPPING : 0) | (process_toast ? VACOPT_PROCESS_TOAST : 0) | (skip_database_stats ? VACOPT_SKIP_DATABASE_STATS : 0) | - (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0); + (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0) | + (process_main ? VACOPT_PROCESS_MAIN : 0); /* sanity checks on options */ Assert(params.options & (VACOPT_VACUUM | VACOPT_ANALYZE)); @@ -367,9 +371,10 @@ vacuum(List *relations, VacuumParams *params, ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("ONLY_DATABASE_STATS cannot be specified with a list of tables"))); - /* don't require people to turn off PROCESS_TOAST explicitly */ + /* don't require people to turn off PROCESS_TOAST/MAIN explicitly */ if (params->options & ~(VACOPT_VACUUM | VACOPT_VERBOSE | + VACOPT_PROCESS_MAIN | VACOPT_PROCESS_TOAST | VACOPT_ONLY_DATABASE_STATS)) ereport(ERROR, @@ -2031,10 +2036,12 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs) /* * Remember the relation's TOAST relation for later, if the caller asked * us to process it. In VACUUM FULL, though, the toast table is - * automatically rebuilt by cluster_rel so we shouldn't recurse to it. + * automatically rebuilt by cluster_rel so we shouldn't recurse to it + * unless PROCESS_MAIN is disabled. */ if ((params->options & VACOPT_PROCESS_TOAST) != 0 && - (params->options & VACOPT_FULL) == 0) + ((params->options & VACOPT_FULL) == 0 || + (params->options & VACOPT_PROCESS_MAIN) == 0)) toast_relid = rel->rd_rel->reltoastrelid; else toast_relid = InvalidOid; @@ -2053,7 +2060,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs) /* * Do the actual work --- either FULL or "lazy" vacuum */ - if (params->options & VACOPT_FULL) + if (params->options & VACOPT_FULL && + params->options & VACOPT_PROCESS_MAIN) { ClusterParams cluster_params = {0}; @@ -2067,7 +2075,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs) /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */ cluster_rel(relid, InvalidOid, &cluster_params); } - else + else if (params->options & VACOPT_PROCESS_MAIN) table_relation_vacuum(rel, params, vac_strategy); /* Roll back any GUC changes executed by index functions */ @@ -2094,7 +2102,15 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs) * totally unimportant for toast relations. */ if (toast_relid != InvalidOid) + { + /* we force VACOPT_PROCESS_MAIN so vacuum_rel() processes it */ + bool force_opt = ((params->options & VACOPT_PROCESS_MAIN) == 0); + + params->options |= VACOPT_PROCESS_MAIN; vacuum_rel(toast_relid, NULL, params, true); + if (force_opt) + params->options &= ~VACOPT_PROCESS_MAIN; + } /* * Now release the session-level lock on the main table. diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c index f5ea381c53..12dcb2b762 100644 --- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -2860,7 +2860,9 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map, * skip vac_update_datfrozenxid(); we'll do that separately. */ tab->at_params.options = - (dovacuum ? (VACOPT_VACUUM | VACOPT_SKIP_DATABASE_STATS) : 0) | + (dovacuum ? (VACOPT_VACUUM | + VACOPT_PROCESS_MAIN | + VACOPT_SKIP_DATABASE_STATS) : 0) | (doanalyze ? VACOPT_ANALYZE : 0) | (!wraparound ? VACOPT_SKIP_LOCKED : 0); diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c index 5e1882eaea..8f12af799b 100644 --- a/src/bin/psql/tab-complete.c +++ b/src/bin/psql/tab-complete.c @@ -4618,10 +4618,10 @@ psql_completion(const char *text, int start, int end) if (ends_with(prev_wd, '(') || ends_with(prev_wd, ',')) COMPLETE_WITH("FULL", "FREEZE", "ANALYZE", "VERBOSE", "DISABLE_PAGE_SKIPPING", "SKIP_LOCKED", - "INDEX_CLEANUP", "PROCESS_TOAST", + "INDEX_CLEANUP", "PROCESS_MAIN", "PROCESS_TOAST", "TRUNCATE", "PARALLEL", "SKIP_DATABASE_STATS", "ONLY_DATABASE_STATS"); - else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS")) + else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_MAIN|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS")) COMPLETE_WITH("ON", "OFF"); else if (TailMatches("INDEX_CLEANUP")) COMPLETE_WITH("AUTO", "ON", "OFF"); diff --git a/src/bin/scripts/t/100_vacuumdb.pl b/src/bin/scripts/t/100_vacuumdb.pl index 3cfbaaec0d..46101899ae 100644 --- a/src/bin/scripts/t/100_vacuumdb.pl +++ b/src/bin/scripts/t/100_vacuumdb.pl @@ -65,6 +65,13 @@ $node->issues_sql_like( $node->command_fails( [ 'vacuumdb', '--analyze-only', '--no-truncate', 'postgres' ], '--analyze-only and --no-truncate specified together'); +$node->issues_sql_like( + [ 'vacuumdb', '--no-process-main', 'postgres' ], + qr/statement: VACUUM \(PROCESS_MAIN FALSE, SKIP_DATABASE_STATS\).*;/, + 'vacuumdb --no-process-main'); +$node->command_fails( + [ 'vacuumdb', '--analyze-only', '--no-process-main', 'postgres' ], + '--analyze-only and --no-process_main specified together'); $node->issues_sql_like( [ 'vacuumdb', '--no-process-toast', 'postgres' ], qr/statement: VACUUM \(PROCESS_TOAST FALSE, SKIP_DATABASE_STATS\).*;/, diff --git a/src/bin/scripts/vacuumdb.c b/src/bin/scripts/vacuumdb.c index 58b894216b..39be265b5b 100644 --- a/src/bin/scripts/vacuumdb.c +++ b/src/bin/scripts/vacuumdb.c @@ -43,6 +43,7 @@ typedef struct vacuumingOptions bool no_index_cleanup; bool force_index_cleanup; bool do_truncate; + bool process_main; bool process_toast; bool skip_database_stats; } vacuumingOptions; @@ -121,6 +122,7 @@ main(int argc, char *argv[]) {"force-index-cleanup", no_argument, NULL, 9}, {"no-truncate", no_argument, NULL, 10}, {"no-process-toast", no_argument, NULL, 11}, + {"no-process-main", no_argument, NULL, 12}, {NULL, 0, NULL, 0} }; @@ -148,6 +150,7 @@ main(int argc, char *argv[]) vacopts.no_index_cleanup = false; vacopts.force_index_cleanup = false; vacopts.do_truncate = true; + vacopts.process_main = true; vacopts.process_toast = true; pg_logging_init(argv[0]); @@ -260,6 +263,9 @@ main(int argc, char *argv[]) case 11: vacopts.process_toast = false; break; + case 12: + vacopts.process_main = false; + break; default: /* getopt_long already emitted a complaint */ pg_log_error_hint("Try \"%s --help\" for more information.", progname); @@ -312,6 +318,9 @@ main(int argc, char *argv[]) if (!vacopts.do_truncate) pg_fatal("cannot use the \"%s\" option when performing only analyze", "no-truncate"); + if (!vacopts.process_main) + pg_fatal("cannot use the \"%s\" option when performing only analyze", + "no-process-main"); if (!vacopts.process_toast) pg_fatal("cannot use the \"%s\" option when performing only analyze", "no-process-toast"); @@ -508,6 +517,13 @@ vacuum_one_database(ConnParams *cparams, "no-truncate", "12"); } + if (!vacopts->process_main && PQserverVersion(conn) < 160000) + { + PQfinish(conn); + pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s", + "no-process-main", "16"); + } + if (!vacopts->process_toast && PQserverVersion(conn) < 140000) { PQfinish(conn); @@ -976,6 +992,13 @@ prepare_vacuum_command(PQExpBuffer sql, int serverVersion, appendPQExpBuffer(sql, "%sTRUNCATE FALSE", sep); sep = comma; } + if (!vacopts->process_main) + { + /* PROCESS_MAIN is supported since v16 */ + Assert(serverVersion >= 160000); + appendPQExpBuffer(sql, "%sPROCESS_MAIN FALSE", sep); + sep = comma; + } if (!vacopts->process_toast) { /* PROCESS_TOAST is supported since v14 */ @@ -1090,6 +1113,7 @@ help(const char *progname) printf(_(" --min-mxid-age=MXID_AGE minimum multixact ID age of tables to vacuum\n")); printf(_(" --min-xid-age=XID_AGE minimum transaction ID age of tables to vacuum\n")); printf(_(" --no-index-cleanup don't remove index entries that point to dead tuples\n")); + printf(_(" --no-process-main skip the main relation\n")); printf(_(" --no-process-toast skip the TOAST table associated with the table to vacuum\n")); printf(_(" --no-truncate don't truncate empty pages at the end of the table\n")); printf(_(" -n, --schema=PATTERN vacuum tables in the specified schema(s) only\n")); diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h index 689dbb7702..29f5c1af04 100644 --- a/src/include/commands/vacuum.h +++ b/src/include/commands/vacuum.h @@ -190,6 +190,7 @@ typedef struct VacAttrStats #define VACOPT_DISABLE_PAGE_SKIPPING 0x80 /* don't skip any pages */ #define VACOPT_SKIP_DATABASE_STATS 0x100 /* skip vac_update_datfrozenxid() */ #define VACOPT_ONLY_DATABASE_STATS 0x200 /* only vac_update_datfrozenxid() */ +#define VACOPT_PROCESS_MAIN 0x400 /* process main relation */ /* * Values used by index_cleanup and truncate params. diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out index 458adee7f8..e482873436 100644 --- a/src/test/regress/expected/vacuum.out +++ b/src/test/regress/expected/vacuum.out @@ -288,6 +288,10 @@ VACUUM (SKIP_DATABASE_STATS) vactst; VACUUM (ONLY_DATABASE_STATS); VACUUM (ONLY_DATABASE_STATS) vactst; -- error ERROR: ONLY_DATABASE_STATS cannot be specified with a list of tables +-- PROCESS_MAIN option +VACUUM (PROCESS_MAIN FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, FULL) vactst; DROP TABLE vaccluster; DROP TABLE vactst; DROP TABLE vacparted; diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql index 9da8f3e830..aaa97aa8ba 100644 --- a/src/test/regress/sql/vacuum.sql +++ b/src/test/regress/sql/vacuum.sql @@ -244,6 +244,11 @@ VACUUM (SKIP_DATABASE_STATS) vactst; VACUUM (ONLY_DATABASE_STATS); VACUUM (ONLY_DATABASE_STATS) vactst; -- error +-- PROCESS_MAIN option +VACUUM (PROCESS_MAIN FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, FULL) vactst; + DROP TABLE vaccluster; DROP TABLE vactst; DROP TABLE vacparted; -- 2.25.1 --4Ckj6UjgE2iN1+kY--