public inbox for [email protected]
help / color / mirror / Atom feedRemove XLogRecGetFullXid() in xlogreader.c?
5+ messages / 4 participants
[nested] [flat]
* Remove XLogRecGetFullXid() in xlogreader.c?
@ 2025-01-17 05:00 Michael Paquier <[email protected]>
2025-01-17 15:39 ` Re: Remove XLogRecGetFullXid() in xlogreader.c? Nathan Bossart <[email protected]>
2025-01-17 15:45 ` Re: Remove XLogRecGetFullXid() in xlogreader.c? Andres Freund <[email protected]>
0 siblings, 2 replies; 5+ messages in thread
From: Michael Paquier @ 2025-01-17 05:00 UTC (permalink / raw)
To: Postgres hackers <[email protected]>
Hi all,
XLogRecGetFullXid() has been introduced in 67b9b3ca3283 back in 2019,
but as far as I can see this has never been used in the code and this
is used nowhere in the core code.
I have looked at Debian's codesearch and also looked at traces of it
on github without seeing it being used anywhere. Knowing that this
was originally intended for a hypothetical undo log patch back then,
for which no work has been done for years, is there any point in
keeping this function in core?
This issue has been raised on a separate thread, where Noah has sent a
patch to consolidate a bit some epoch calculations for
FullTransactionIds, around here:
https://www.postgresql.org/message-id/[email protected]
Removing it would have the benefit to do a bit less refactoring for
some of the work of the other thread, and this removes all traces of
-DFRONTEND in xlogreader.h. ;)
Thoughts or comments?
--
Michael
Attachments:
[text/x-diff] 0001-Remove-XLogRecGetFullXid.patch (2.8K, ../../[email protected]/2-0001-Remove-XLogRecGetFullXid.patch)
download | inline diff:
From 101b8469dff3f35c9c3bcd24f1bcfff740f3a622 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Fri, 17 Jan 2025 13:56:40 +0900
Subject: [PATCH] Remove XLogRecGetFullXid()
---
src/include/access/xlogreader.h | 8 ------
src/include/storage/standby.h | 1 +
src/backend/access/transam/xlogreader.c | 34 -------------------------
3 files changed, 1 insertion(+), 42 deletions(-)
diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h
index 9738462d3c..18a67acf42 100644
--- a/src/include/access/xlogreader.h
+++ b/src/include/access/xlogreader.h
@@ -34,10 +34,6 @@
#ifndef XLOGREADER_H
#define XLOGREADER_H
-#ifndef FRONTEND
-#include "access/transam.h"
-#endif
-
#include "access/xlogrecord.h"
#include "storage/buf.h"
@@ -427,10 +423,6 @@ extern bool DecodeXLogRecord(XLogReaderState *state,
#define XLogRecHasBlockData(decoder, block_id) \
((decoder)->record->blocks[block_id].has_data)
-#ifndef FRONTEND
-extern FullTransactionId XLogRecGetFullXid(XLogReaderState *record);
-#endif
-
extern bool RestoreBlockImage(XLogReaderState *record, uint8 block_id, char *page);
extern char *XLogRecGetBlockData(XLogReaderState *record, uint8 block_id, Size *len);
extern void XLogRecGetBlockTag(XLogReaderState *record, uint8 block_id,
diff --git a/src/include/storage/standby.h b/src/include/storage/standby.h
index 24e2f5082b..995d3c365d 100644
--- a/src/include/storage/standby.h
+++ b/src/include/storage/standby.h
@@ -14,6 +14,7 @@
#ifndef STANDBY_H
#define STANDBY_H
+#include "access/transam.h"
#include "datatype/timestamp.h"
#include "storage/lock.h"
#include "storage/procsignal.h"
diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c
index 3596af0617..12d5175dc3 100644
--- a/src/backend/access/transam/xlogreader.c
+++ b/src/backend/access/transam/xlogreader.c
@@ -2157,37 +2157,3 @@ RestoreBlockImage(XLogReaderState *record, uint8 block_id, char *page)
return true;
}
-
-#ifndef FRONTEND
-
-/*
- * Extract the FullTransactionId from a WAL record.
- */
-FullTransactionId
-XLogRecGetFullXid(XLogReaderState *record)
-{
- TransactionId xid,
- next_xid;
- uint32 epoch;
-
- /*
- * This function is only safe during replay, because it depends on the
- * replay state. See AdvanceNextFullTransactionIdPastXid() for more.
- */
- Assert(AmStartupProcess() || !IsUnderPostmaster);
-
- xid = XLogRecGetXid(record);
- next_xid = XidFromFullTransactionId(TransamVariables->nextXid);
- epoch = EpochFromFullTransactionId(TransamVariables->nextXid);
-
- /*
- * If xid is numerically greater than next_xid, it has to be from the last
- * epoch.
- */
- if (unlikely(xid > next_xid))
- --epoch;
-
- return FullTransactionIdFromEpochAndXid(epoch, xid);
-}
-
-#endif
--
2.47.1
[application/pgp-signature] signature.asc (833B, ../../[email protected]/3-signature.asc)
download
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Remove XLogRecGetFullXid() in xlogreader.c?
2025-01-17 05:00 Remove XLogRecGetFullXid() in xlogreader.c? Michael Paquier <[email protected]>
@ 2025-01-17 15:39 ` Nathan Bossart <[email protected]>
1 sibling, 0 replies; 5+ messages in thread
From: Nathan Bossart @ 2025-01-17 15:39 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Postgres hackers <[email protected]>; [email protected]
(I've added Thomas Munro to the thread.)
On Fri, Jan 17, 2025 at 02:00:49PM +0900, Michael Paquier wrote:
> XLogRecGetFullXid() has been introduced in 67b9b3ca3283 back in 2019,
> but as far as I can see this has never been used in the code and this
> is used nowhere in the core code.
>
> I have looked at Debian's codesearch and also looked at traces of it
> on github without seeing it being used anywhere. Knowing that this
> was originally intended for a hypothetical undo log patch back then,
> for which no work has been done for years, is there any point in
> keeping this function in core?
>
> This issue has been raised on a separate thread, where Noah has sent a
> patch to consolidate a bit some epoch calculations for
> FullTransactionIds, around here:
> https://www.postgresql.org/message-id/[email protected]
>
> Removing it would have the benefit to do a bit less refactoring for
> some of the work of the other thread, and this removes all traces of
> -DFRONTEND in xlogreader.h. ;)
Seems reasonable to me. I think the counterargument is that folks
developing new AMs should use this [0], but if no such users have
materialized in several years, then maybe that's no longer a concern.
[0] https://postgr.es/m/CA%2BhUKG%2BmLmuDjMi6o1dxkKvGRL56Y2Rz%2BiXAcrZV03G9ZuFQ8Q%40mail.gmail.com
--
nathan
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Remove XLogRecGetFullXid() in xlogreader.c?
2025-01-17 05:00 Remove XLogRecGetFullXid() in xlogreader.c? Michael Paquier <[email protected]>
@ 2025-01-17 15:45 ` Andres Freund <[email protected]>
2025-01-17 23:21 ` Re: Remove XLogRecGetFullXid() in xlogreader.c? Michael Paquier <[email protected]>
1 sibling, 1 reply; 5+ messages in thread
From: Andres Freund @ 2025-01-17 15:45 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Postgres hackers <[email protected]>
Hi,
On 2025-01-17 14:00:49 +0900, Michael Paquier wrote:
> XLogRecGetFullXid() has been introduced in 67b9b3ca3283 back in 2019,
> but as far as I can see this has never been used in the code and this
> is used nowhere in the core code.
>
> I have looked at Debian's codesearch and also looked at traces of it
> on github without seeing it being used anywhere. Knowing that this
> was originally intended for a hypothetical undo log patch back then,
> for which no work has been done for years, is there any point in
> keeping this function in core?
>
> This issue has been raised on a separate thread, where Noah has sent a
> patch to consolidate a bit some epoch calculations for
> FullTransactionIds, around here:
> https://www.postgresql.org/message-id/[email protected]
>
> Removing it would have the benefit to do a bit less refactoring for
> some of the work of the other thread, and this removes all traces of
> -DFRONTEND in xlogreader.h. ;)
-0.5
I think it is sensible infrastructure and we'll eventually end up using
it. The cost of having it fairly low. If it were a legacy thing that we
shouldn't introduce new users for it'd be a different story, but it's the
opposite.
Greetings,
Andres
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Remove XLogRecGetFullXid() in xlogreader.c?
2025-01-17 05:00 Remove XLogRecGetFullXid() in xlogreader.c? Michael Paquier <[email protected]>
2025-01-17 15:45 ` Re: Remove XLogRecGetFullXid() in xlogreader.c? Andres Freund <[email protected]>
@ 2025-01-17 23:21 ` Michael Paquier <[email protected]>
0 siblings, 0 replies; 5+ messages in thread
From: Michael Paquier @ 2025-01-17 23:21 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Postgres hackers <[email protected]>
On Fri, Jan 17, 2025 at 10:45:28AM -0500, Andres Freund wrote:
> I think it is sensible infrastructure and we'll eventually end up using
> it. The cost of having it fairly low. If it were a legacy thing that we
> shouldn't introduce new users for it'd be a different story, but it's the
> opposite.
The last 6 years are showing that we are still looking for such users,
so honestly, I am not sure if it's worth keeping. Let's see if others
have an opinion. I'm OK if the consensus is to let things be, as
well.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 5+ messages in thread
* [PATCH] Fix a few problems in index build progress reporting.
@ 2026-03-16 12:53 Antonin Houska <[email protected]>
0 siblings, 0 replies; 5+ messages in thread
From: Antonin Houska @ 2026-03-16 12:53 UTC (permalink / raw)
First, index_build() should not update the progress when being driven by
REPACK, because the progress reporting infractructure cannot handle status of
two commands at the same time. So far, REPACK with the CONCURRENTLY option
neglected this problem altogether, but even the existing REPACK wasn't
consistent enough: even if the 'progress' variable in repack_index() was
false, it didn't pass the value to index_build().
Second, REPACK (CONCURRENTLY) should not set PROGRESS_REPACK_PHASE to
PROGRESS_REPACK_PHASE_FINAL_CLEANUP in rebuild_relation() because it calls
finish_heap_swap() anyway (via rebuild_relation_finish_concurrent()), which
does the same thing.
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/catalog/heap.c | 3 ++-
src/backend/catalog/index.c | 22 ++++++++++++++++++----
src/backend/catalog/toasting.c | 3 ++-
src/backend/commands/cluster.c | 7 ++++---
src/backend/commands/indexcmds.c | 1 +
src/include/catalog/index.h | 4 +++-
7 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 17118f2fe76..cb06c52c1d4 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -1180,7 +1180,7 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false);
+ index_build(heap, ind, ILHead->il_info, false, false, false);
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5748aa9a1a9..ae6b7cda3dd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3570,7 +3570,8 @@ RelationTruncateIndexes(Relation heapRelation)
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false);
+ index_build(heapRelation, currentIndex, indexInfo, true, false,
+ true);
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b1e05d56a1e..89aad2f7b77 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -714,6 +714,9 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
+ * INDEX_CREATE_REPORT_PROGRESS:
+ * update the backend's progress information during index build.
+
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
* allow_system_table_mods: allow table to be a system catalog
@@ -759,6 +762,7 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
+ bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1275,7 +1279,8 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true);
+ index_build(heapRelation, indexRelation, indexInfo, false, true,
+ progress);
}
/*
@@ -1454,6 +1459,12 @@ index_create_copy(Relation heapRelation, Oid oldIndexId, Oid tablespaceOid,
stattargets[i].isnull = isnull;
}
+ /*
+ * Note: The current callers do not need INDEX_CREATE_REPORT_PROGRESS. If
+ * 'concurrently' is true, there is no build at all. Otherwise the index
+ * build is a sub-command of REPACK. The current infrastructure does not
+ * allow two commands to report their progress at the same time.
+ */
if (concurrently)
flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
@@ -1544,7 +1555,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -3017,6 +3028,7 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
+ * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3027,7 +3039,8 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel)
+ bool parallel,
+ bool progress)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3078,6 +3091,7 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
+ if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3835,7 +3849,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true);
+ index_build(heapRelation, iRel, indexInfo, true, true, progress);
/* Re-allow use of target index */
ResetReindexProcessing();
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 078a1cf5127..73c41360ae9 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -331,7 +331,8 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
BTREE_AM_OID,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
- INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,
+ true, true, NULL);
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index fe419ef808e..9aaa8e95389 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1162,9 +1162,6 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose,
rebuild_relation_finish_concurrent(NewHeap, OldHeap, ident_idx,
frozenXid, cutoffMulti);
-
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
}
else
{
@@ -3505,6 +3502,10 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
relpersistence = OldHeap->rd_rel->relpersistence;
is_system_catalog = IsSystemRelation(OldHeap);
+ /*
+ * finish_heap_swap() sets the phase too, but here we need some swapping
+ * in advance.
+ */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 34209bd1393..ad5c1238e09 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1230,6 +1230,7 @@ DefineIndex(ParseState *pstate,
flags |= INDEX_CREATE_PARTITIONED;
if (stmt->primary)
flags |= INDEX_CREATE_IS_PRIMARY;
+ flags |= INDEX_CREATE_REPORT_PROGRESS;
/*
* If the table is partitioned, and recursion was declined but partitions
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 3426087b445..3778e9b7f07 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -65,6 +65,7 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
+#define INDEX_CREATE_REPORT_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -146,7 +147,8 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel);
+ bool parallel,
+ bool progress);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2026-03-16 12:53 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-01-17 05:00 Remove XLogRecGetFullXid() in xlogreader.c? Michael Paquier <[email protected]>
2025-01-17 15:39 ` Nathan Bossart <[email protected]>
2025-01-17 15:45 ` Andres Freund <[email protected]>
2025-01-17 23:21 ` Michael Paquier <[email protected]>
2026-03-16 12:53 [PATCH] Fix a few problems in index build progress reporting. Antonin Houska <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox