From: Álvaro Herrera <alvherre@alvh.no-ip.org>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Hamid Akhtar <hamid.akhtar@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: {CREATE INDEX, REINDEX} CONCURRENTLY improvements
Date: Mon, 22 Feb 2021 21:15:31 -0300
Message-ID: <20210223001531.GA29197@alvherre.pgsql> (raw)
In-Reply-To: <20210222155642.GA26534@alvherre.pgsql>
On 2021-Feb-22, Álvaro Herrera wrote:
> Here's an updated patch.
>
> I haven't added commentary or documentation to account for the new
> assumption, per Matthias' comment and Robert's discussion thereof.
Done that. I also restructured the code a bit, since one line in
ComputedXidHorizons was duplicative.
--
Álvaro Herrera 39°49'30"S 73°17'W
Tom: There seems to be something broken here.
Teodor: I'm in sackcloth and ashes... Fixed.
http://archives.postgresql.org/message-id/482D1632.8010507@sigaev.ruAttachments:v3-0001-Teach-VACUUM-to-ignore-CIC.patch (text/x-diff, 5.2K) download
From 98fa9ae33cb890e1d24a0c9be3a14139f15fdafe Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 15 Jan 2021 14:03:14 -0300
Subject: [PATCH v3] Teach VACUUM to ignore CIC
Discussion: https://postgr.es/m/20210115133858.GA18931@alvherre.pgsql
---
src/backend/storage/ipc/procarray.c | 39 +++++++++++++++++++++++------
src/backend/utils/misc/guc.c | 2 +-
2 files changed, 32 insertions(+), 9 deletions(-)
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.cindex a5daea8957..9c88f58c7f 100644--- a/src/backend/storage/ipc/procarray.c+++ b/src/backend/storage/ipc/procarray.c@@ -1610,7 +1610,13 @@ TransactionIdIsActive(TransactionId xid)
* relations that's not required, since only backends in my own database could
* ever see the tuples in them. Also, we can ignore concurrently running lazy
* VACUUMs because (a) they must be working on other tables, and (b) they
- * don't need to do snapshot-based lookups.+ * don't need to do snapshot-based lookups. Similarly, for the non-catalog+ * horizon, we can ignore CREATE INDEX CONCURRENTLY and REINDEX CONCURRENTLY+ * when they are working on non-partial, non-expressional indexes, for the+ * same reasons and because they can't run in transaction blocks. (They are+ * not possible to ignore for catalogs, because CIC and RC do some catalog+ * operations.) Do note that this means that CIC and RC must use a lock level+ * that conflicts with VACUUM.
*
* This also computes a horizon used to truncate pg_subtrans. For that
* backends in all databases have to be considered, and concurrently running
@@ -1660,9 +1666,6 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
bool in_recovery = RecoveryInProgress();
TransactionId *other_xids = ProcGlobal->xids;
- /* inferred after ProcArrayLock is released */- h->catalog_oldest_nonremovable = InvalidTransactionId;-
LWLockAcquire(ProcArrayLock, LW_SHARED);
h->latest_completed = ShmemVariableCache->latestCompletedXid;
@@ -1682,6 +1685,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
h->oldest_considered_running = initial;
h->shared_oldest_nonremovable = initial;
+ h->catalog_oldest_nonremovable = initial;
h->data_oldest_nonremovable = initial;
/*
@@ -1752,7 +1756,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
if (statusFlags & (PROC_IN_VACUUM | PROC_IN_LOGICAL_DECODING))
continue;
- /* shared tables need to take backends in all database into account */+ /* shared tables need to take backends in all databases into account */
h->shared_oldest_nonremovable =
TransactionIdOlder(h->shared_oldest_nonremovable, xmin);
@@ -1773,11 +1777,26 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
MyDatabaseId == InvalidOid || proc->databaseId == MyDatabaseId ||
proc->databaseId == 0) /* always include WalSender */
{
- h->data_oldest_nonremovable =- TransactionIdOlder(h->data_oldest_nonremovable, xmin);+ /*+ * We can ignore this backend if it's running CREATE INDEX+ * CONCURRENTLY or REINDEX CONCURRENTLY on a "safe" index -- but+ * only on vacuums of user-defined tables.+ */+ if (!(statusFlags & PROC_IN_SAFE_IC))+ h->data_oldest_nonremovable =+ TransactionIdOlder(h->data_oldest_nonremovable, xmin);++ /* Catalog tables need to consider all backends. */+ h->catalog_oldest_nonremovable =+ TransactionIdOlder(h->catalog_oldest_nonremovable, xmin);+
}
}
+ /* catalog horizon should never be later than data */+ Assert(TransactionIdPrecedesOrEquals(h->catalog_oldest_nonremovable,+ h->data_oldest_nonremovable));+
/*
* If in recovery fetch oldest xid in KnownAssignedXids, will be applied
* after lock is released.
@@ -1799,6 +1818,8 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
TransactionIdOlder(h->shared_oldest_nonremovable, kaxmin);
h->data_oldest_nonremovable =
TransactionIdOlder(h->data_oldest_nonremovable, kaxmin);
+ h->catalog_oldest_nonremovable =+ TransactionIdOlder(h->catalog_oldest_nonremovable, kaxmin);
/* temp relations cannot be accessed in recovery */
}
else
@@ -1825,6 +1846,9 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
h->data_oldest_nonremovable =
TransactionIdRetreatedBy(h->data_oldest_nonremovable,
vacuum_defer_cleanup_age);
+ h->catalog_oldest_nonremovable =+ TransactionIdRetreatedBy(h->catalog_oldest_nonremovable,+ vacuum_defer_cleanup_age);
/* defer doesn't apply to temp relations */
}
@@ -1847,7 +1871,6 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
h->shared_oldest_nonremovable =
TransactionIdOlder(h->shared_oldest_nonremovable,
h->slot_catalog_xmin);
- h->catalog_oldest_nonremovable = h->data_oldest_nonremovable;
h->catalog_oldest_nonremovable =
TransactionIdOlder(h->catalog_oldest_nonremovable,
h->slot_catalog_xmin);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.cindex 260ec7b97e..d626731723 100644--- a/src/backend/utils/misc/guc.c+++ b/src/backend/utils/misc/guc.c@@ -2583,7 +2583,7 @@ static struct config_int ConfigureNamesInt[] =
NULL
},
&vacuum_defer_cleanup_age,
- 0, 0, 1000000,+ 0, 0, 1000000, /* see ComputeXidHorizons */
NULL, NULL, NULL
},
--
2.20.1
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: alvherre@alvh.no-ip.org, boekewurm+postgres@gmail.com, sawada.mshk@gmail.com, hamid.akhtar@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: {CREATE INDEX, REINDEX} CONCURRENTLY improvements
In-Reply-To: <20210223001531.GA29197@alvherre.pgsql>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox