From: Álvaro Herrera <alvherre@alvh.no-ip.org>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Hamid Akhtar <hamid.akhtar@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: {CREATE INDEX, REINDEX} CONCURRENTLY improvements
Date: Fri, 15 Jan 2021 11:29:26 -0300
Message-ID: <20210115142926.GA19300@alvherre.pgsql> (raw)
In-Reply-To: <20210115133858.GA18931@alvherre.pgsql>
So one last remaining improvement was to have VACUUM ignore processes
doing CIC and RC to compute the Xid horizon of tuples to remove. I
think we can do something simple like the attached patch.
--
Álvaro Herrera Valdivia, Chile
"Aprender sin pensar es inútil; pensar sin aprender, peligroso" (Confucio)
Attachments:vacuum-cic.patch (text/x-diff, 4.9K) downloaddiff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.cindex cf12eda504..f584230b79 100644--- a/src/backend/storage/ipc/procarray.c+++ b/src/backend/storage/ipc/procarray.c@@ -1601,7 +1601,7 @@ TransactionIdIsActive(TransactionId xid)
* well as "internally" by GlobalVisUpdate() (see comment above struct
* GlobalVisState).
*
- * See the definition of ComputedXidHorizonsResult for the various computed+ * See the definition of ComputeXidHorizonsResult for the various computed
* horizons.
*
* For VACUUM separate horizons (used to decide which deleted tuples must
@@ -1610,7 +1610,12 @@ TransactionIdIsActive(TransactionId xid)
* relations that's not required, since only backends in my own database could
* ever see the tuples in them. Also, we can ignore concurrently running lazy
* VACUUMs because (a) they must be working on other tables, and (b) they
- * don't need to do snapshot-based lookups.+ * don't need to do snapshot-based lookups. Also, for the non-catalog+ * horizon, we can ignore CREATE INDEX CONCURRENTLY and REINDEX CONCURRENTLY+ * when they are working on non-partial, non-expressional indexes, for the+ * same reasons and because they can't run in transaction blocks. (They are+ * not possible to ignore for catalogs, because CIC and RC do some catalog+ * operations.)
*
* This also computes a horizon used to truncate pg_subtrans. For that
* backends in all databases have to be considered, and concurrently running
@@ -1660,9 +1665,6 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
bool in_recovery = RecoveryInProgress();
TransactionId *other_xids = ProcGlobal->xids;
- /* inferred after ProcArrayLock is released */- h->catalog_oldest_nonremovable = InvalidTransactionId;-
LWLockAcquire(ProcArrayLock, LW_SHARED);
h->latest_completed = ShmemVariableCache->latestCompletedXid;
@@ -1682,6 +1684,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
h->oldest_considered_running = initial;
h->shared_oldest_nonremovable = initial;
+ h->catalog_oldest_nonremovable = initial;
h->data_oldest_nonremovable = initial;
/*
@@ -1752,7 +1755,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
if (statusFlags & (PROC_IN_VACUUM | PROC_IN_LOGICAL_DECODING))
continue;
- /* shared tables need to take backends in all database into account */+ /* shared tables need to take backends in all databases into account */
h->shared_oldest_nonremovable =
TransactionIdOlder(h->shared_oldest_nonremovable, xmin);
@@ -1768,16 +1771,29 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
* to prune still needed data away). If the current backend never
* connects to a database that is harmless, because
* data_oldest_nonremovable will never be utilized.
+ *+ * Additionally, processes doing CREATE INDEX CONCURRENTLY and REINDEX+ * CONCURRENTLY on "safe" indexes can be ignored for non-catalog+ * horizon. (But not for catalogs: some transactions in CIC/RC do+ * catalog updates.)
*/
if (in_recovery ||
MyDatabaseId == InvalidOid || proc->databaseId == MyDatabaseId ||
proc->databaseId == 0) /* always include WalSender */
{
- h->data_oldest_nonremovable =- TransactionIdOlder(h->data_oldest_nonremovable, xmin);+ if (statusFlags & PROC_IN_SAFE_IC)+ h->catalog_oldest_nonremovable =+ TransactionIdOlder(h->catalog_oldest_nonremovable, xmin);+ else+ h->data_oldest_nonremovable = h->catalog_oldest_nonremovable =+ TransactionIdOlder(h->data_oldest_nonremovable, xmin);
}
}
+ /* catalog horizon should never be later than data */+ Assert(TransactionIdPrecedesOrEquals(h->catalog_oldest_nonremovable,+ h->data_oldest_nonremovable));+
/*
* If in recovery fetch oldest xid in KnownAssignedXids, will be applied
* after lock is released.
@@ -1799,6 +1815,8 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
TransactionIdOlder(h->shared_oldest_nonremovable, kaxmin);
h->data_oldest_nonremovable =
TransactionIdOlder(h->data_oldest_nonremovable, kaxmin);
+ h->catalog_oldest_nonremovable =+ TransactionIdOlder(h->catalog_oldest_nonremovable, kaxmin);
/* temp relations cannot be accessed in recovery */
}
else
@@ -1825,6 +1843,9 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
h->data_oldest_nonremovable =
TransactionIdRetreatedBy(h->data_oldest_nonremovable,
vacuum_defer_cleanup_age);
+ h->catalog_oldest_nonremovable =+ TransactionIdRetreatedBy(h->catalog_oldest_nonremovable,+ vacuum_defer_cleanup_age);
/* defer doesn't apply to temp relations */
}
@@ -1847,7 +1868,6 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
h->shared_oldest_nonremovable =
TransactionIdOlder(h->shared_oldest_nonremovable,
h->slot_catalog_xmin);
- h->catalog_oldest_nonremovable = h->data_oldest_nonremovable;
h->catalog_oldest_nonremovable =
TransactionIdOlder(h->catalog_oldest_nonremovable,
h->slot_catalog_xmin);
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: alvherre@alvh.no-ip.org, sawada.mshk@gmail.com, hamid.akhtar@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: {CREATE INDEX, REINDEX} CONCURRENTLY improvements
In-Reply-To: <20210115142926.GA19300@alvherre.pgsql>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox